-- Logs begin at Wed 2022-08-24 16:56:58 UTC, end at Wed 2022-08-24 18:10:43 UTC. -- Aug 24 17:28:14.230126 dv-cinder-854428-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 17:28:14.237635 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 17:28:14 2022] *** Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: nodename: dv-cinder-854428-1-iscsi Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: machine: x86_64 Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: clock source: unix Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: pcre jit disabled Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: detected number of CPU cores: 4 Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: current working directory: / Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: detected binary path: /usr/bin/uwsgi-core Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: your processes number limit is 47680 Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: your memory page size is 4096 bytes Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: detected max file descriptor number: 2048 Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: lock engine: pthread robust mutexes Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: thunder lock: enabled Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 17:28:14.241885 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 17:28:14.282974 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: Python main interpreter initialized at 0x55f1352f39c0 Aug 24 17:28:14.282974 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: python threads support enabled Aug 24 17:28:14.282974 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: your server socket listen backlog is limited to 100 connections Aug 24 17:28:14.282974 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: your mercy for graceful operations on workers is 90 seconds Aug 24 17:28:14.283366 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 17:28:14.283445 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: *** Operational MODE: preforking *** Aug 24 17:28:14.283693 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: *** uWSGI is running in multiple interpreter mode *** Aug 24 17:28:14.283693 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: spawned uWSGI master process (pid: 81873) Aug 24 17:28:14.283908 dv-cinder-854428-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 17:28:14.285030 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: spawned uWSGI worker 1 (pid: 81886, cores: 1) Aug 24 17:28:14.285030 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: spawned uWSGI worker 2 (pid: 81887, cores: 1) Aug 24 17:28:14.285030 dv-cinder-854428-1-iscsi devstack@keystone.service[81873]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 17:28:18.714937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:28:18.714937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: from cryptography.utils import int_from_bytes Aug 24 17:28:18.715868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:28:18.715868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: from cryptography.utils import int_from_bytes Aug 24 17:28:18.716369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:28:18.716369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: from cryptography.utils import int_from_bytes Aug 24 17:28:18.716531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:28:18.716531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: from cryptography.utils import int_from_bytes Aug 24 17:28:18.766573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 17:28:18.766769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 17:28:18.766846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 17:28:18.766955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 17:28:18.767038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] command line args: [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 17:28:18.767113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] command line args: [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 17:28:18.767186 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 17:28:18.767404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 17:28:18.767542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ================================================================================ {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 17:28:18.767675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ================================================================================ {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 17:28:18.767784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] admin_token = **** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.767888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] admin_token = **** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.767988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] config_dir = [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768057 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] config_dir = [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] config_source = [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] config_source = [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] debug = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] debug = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.768903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.769003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.769098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.769178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.769308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.769374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.769440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.769622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.769709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] list_limit = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] list_limit = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] log_config_append = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] log_config_append = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] log_dir = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] log_dir = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] log_file = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] log_file = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.770866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.771048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.771132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.771313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.771425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.772152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.772250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.772333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.772509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.772634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.772711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.772775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.772877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773256 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.773978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.774069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.774229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.774317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.774476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.774559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.774755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.774843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.775283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.76/identity {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.775376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.76/identity {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.775458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] publish_errors = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.775582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] publish_errors = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.775688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.775768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.775921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776447 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] standard_threads = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.776890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] standard_threads = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777533 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] use_journal = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] use_journal = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] use_json = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.777942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] use_json = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.778025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] use_stderr = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.778162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] use_syslog = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.778276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] use_stderr = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.778435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.778628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] use_syslog = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.778871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.778952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:28:18.779025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.779097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.779176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.779340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.779432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.779589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.779673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.779834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.779914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.780084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.780173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.780322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.780408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.780573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.780653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.780814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.780914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.781069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.781161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.781303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.781389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.781550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.781642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.781787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.781890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.782061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.782146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.782304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.782391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.782540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.782625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.782798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.782876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.783043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.783124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.783289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.783368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.783526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.783606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.783783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.783864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784859 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.784994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.785078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.785222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.785304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.785458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.785539 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.785710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.785789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.785985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.786069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.786219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.786299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.786453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.786533 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.786702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.786782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.786937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.787016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.787205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.787289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.787458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.787545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.787831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.787896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.788972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.789973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.790999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.791141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.791221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.791379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.791460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.791619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.791711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.791858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.791940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.792096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.792180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.792335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.792422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.792566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.792652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.792807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.792929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.793044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.793132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.793281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.793360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.793516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.793652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.793778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.793927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.798889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799227 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799227 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799451 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.799963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.800992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.801929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.802536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.803981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] auth.external = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.804915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] auth.password = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805533 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] auth.token = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] auth.external = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.805940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] auth.password = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] auth.token = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806679 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] credential.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.806958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.807892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] credential.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.808065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.808147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.808319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.808398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.808557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.808689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.808803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.808890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.809055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.809137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.809298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.809388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.809559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.809655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.809814 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.809914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.810080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.810241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.810321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] federation.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.810393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.810547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.810644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.810820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.810910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.811071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.811152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.811304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.811394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.811555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.811646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.811798 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.811905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] federation.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.812051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.812132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.812291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.812387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.812544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.812625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.812782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.812862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.813032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.813126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.813269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.813351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.813519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.813600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.813760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.813885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.814986 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.815067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.815238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.815326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.815481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.815562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.815714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.815795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.815950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.816053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.816193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.816264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.816439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.816521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.816969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.817066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.817198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.817286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.817438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.817519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.817678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.817937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.818055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.818189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.818869 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.818964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.819165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.819305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.819545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.819883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.820141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.820228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.820379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.820462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.820620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.820716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.820868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.820950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.821111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.821194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.821363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.821444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.821600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.821680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.821839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.821956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.822109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.822242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.822346 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.822428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.822597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.822679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.822852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.823091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.823211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.823346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.823507 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.823615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.823752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.823836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.823990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.824072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.824234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.824318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.824476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.824557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.824757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.824875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.824995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.825121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.825239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.825365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.825482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.825619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.825893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.825978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.826134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.826216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.826371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.826463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.826625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.826708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.826876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.826958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.827115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.827195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.827353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.827433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.827590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.827671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.827850 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.827970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.828082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.828166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.828323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.828405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.828567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.828653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.828805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.828923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.829052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.829134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.829300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.829380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.829538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.829629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.829771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.829938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.830995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.831078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.831236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.831317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.831479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.831561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.831718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.831800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.831963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.832043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.832198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.832279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.832446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.832529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.832687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.832768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.832927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.833009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.833162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.833243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.833400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.833486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.833645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.833725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.833909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.834040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.834767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.834856 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.835066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.835168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.835920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.836934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837007 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] resource.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.837951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.838087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.838198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.838329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.838444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] resource.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.838584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.838702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.838840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.838958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.839078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.839204 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.839326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.839461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.839552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.839719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.839801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.839961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.840042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] role.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.840212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.840301 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] role.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.840456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.840545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.840699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.840780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.840941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] role.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.841034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.841194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] role.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.841276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.841431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.841513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.841679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.841772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.841955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.842039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.842207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.842288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.842459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.842551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.842696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.842776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.842948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.843029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.843197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.843298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.843478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.843572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.843691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.843775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.843945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.844133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.844216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.844378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.844459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.844632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.844723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.844886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.845096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.845177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.845340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.845421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.845597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.845680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.845889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.845972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.846128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.846210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.846370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.846453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.846607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.846687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.846839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.846920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.847075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.847158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.847332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.847424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.847564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.847645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.847804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.847886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] token.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.848996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.849078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.849244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.849332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.849481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.849564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.849784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] token.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.849926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.850057 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.850191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.850315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.850432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.850558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.850680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.850821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.850926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.851078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.851160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.851321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.851402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.851572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.851654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.851822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.851903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.852081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.852163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.852326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.852408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.852555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.852635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.852797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.852879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.853049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.853131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.853303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.853392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.connection = **** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.853539 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.853629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.853793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.853948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.854069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.854150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.854314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.854396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.connection = **** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.854552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.854643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.854804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.854928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855373 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.855998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.856079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.856232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.856313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.856478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.856557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.856713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.856793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.856960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.857042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.857191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.857270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.857431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.857513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.857669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.857751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.857826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 17:28:18.857966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.858166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.858414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.858657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:28:18.858831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81886) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 17:28:18.887823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.888098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.888330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.888755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.888991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.889219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.897457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.898351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.928691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.928884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.928959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.929026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.991506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:18.991696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.041178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.041417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.041978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.042223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.206663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.206870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.207102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.207324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.207481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.207573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.207742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.207834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.207973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.208198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.208423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.208646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.232929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.233997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.234076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.234209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.234337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.252004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.252206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.266349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.266536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.266609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.266672 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.266792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.266879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.267025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.267147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.267232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.267321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.267444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.267575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.267661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.267738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81886) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:28:19.285710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:28:19.292083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:28:19.292400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:28:19.292490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:28:19.292718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.292918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.293054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.293054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.298101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.298404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.298681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.298974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.299078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.299276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.299563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.299860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.306209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.307314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.309163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.309449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.309737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.310051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.310334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.310427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.310598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.310697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.310877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.310968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.311154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.311238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.311451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.311541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.311720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.311823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.311998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.312100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.312282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.312681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.312968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.313349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.336145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.337551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.337652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.337971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.339182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.339479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.342812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.343114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.343405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.344390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.344696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.344992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.350462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.350784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.352367 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.352671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.354922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.355217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.355510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.356311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.356597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.356878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.357034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.357188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.357271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.357421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.357506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.357953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.358112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.358376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.358659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.358937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.375499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.380244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.380459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.380459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.380616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.382528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.384014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.384337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.387131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.387447 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.388709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.391986 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.393463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.395923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.395923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.397155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.397155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.397155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.397498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.398220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.401575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.401949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.402246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.402554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.402881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.403133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.413705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.414057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.414337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.414609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.419683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.420014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.420294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.420593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.421636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.423213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.423497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.426184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.426509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.426777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.427073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.427359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.427812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.429541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.429823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.432554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.432849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.433161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.433451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.433739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.439014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.439314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.439603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.439888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.440174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.440455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.440735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.446313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.446620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.446914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.447226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.447519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.447806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.448094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.457380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.461214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.464206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.464598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.465661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.467180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.467468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.467765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.467914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.470885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.475198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.475198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.475198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.475569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.475725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.476119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.479262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.479587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.479876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.480168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.481549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.483147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.486302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.486620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.486915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.487206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.488390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.488709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.489011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.495795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.496098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.496273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.496423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.496899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.496899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.497143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.497437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.497730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.498054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.498365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.498645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.498940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.499218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.499509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.499789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.500056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.509171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81886) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:28:19.509171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.509171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.509171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.509171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.509171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.509171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.510398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.510398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.510398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.510398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.510398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.510398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.511238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81886) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:28:19.536228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f1352f39c0 pid: 81887 (default app) Aug 24 17:28:19.540510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6886e-74b8-47f5-bdbf-c0d469238059 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.540992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6886e-74b8-47f5-bdbf-c0d469238059 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.541208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6886e-74b8-47f5-bdbf-c0d469238059 None None] PATH_INFO: `/v3` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.545286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f1352f39c0 pid: 81886 (default app) Aug 24 17:28:19.545634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1/1] 10.222.0.76 () {50 vars in 791 bytes} [Wed Aug 24 17:28:19 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:27.541339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1227622-109a-44a1-8780-e93f6e2d1828 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.541948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1227622-109a-44a1-8780-e93f6e2d1828 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.541948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1227622-109a-44a1-8780-e93f6e2d1828 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.546323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1/2] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:27.556489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.557030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.557187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.630413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: INFO dbcounter [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] Registered counter for database keystone Aug 24 17:28:27.652201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG oslo_db.sqlalchemy.engines [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81887) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 17:28:27.659453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writer thread running {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 17:28:27.849899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG passlib.handlers.bcrypt [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81887) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 17:28:27.850239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG passlib.handlers.bcrypt [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81887) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 17:28:27.909157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:27.910897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:28.060048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 2/3] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 17:28:28.637026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1227622-109a-44a1-8780-e93f6e2d1828 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.709009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: INFO dbcounter [None req-d1227622-109a-44a1-8780-e93f6e2d1828 None None] Registered counter for database keystone Aug 24 17:28:28.730939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG oslo_db.sqlalchemy.engines [None req-d1227622-109a-44a1-8780-e93f6e2d1828 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81886) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 17:28:28.737843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writer thread running {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 17:28:28.866799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1227622-109a-44a1-8780-e93f6e2d1828 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.871970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209505ab-72f8-413e-90ca-f9f423c02d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.872662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209505ab-72f8-413e-90ca-f9f423c02d38 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.872945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209505ab-72f8-413e-90ca-f9f423c02d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.873218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209505ab-72f8-413e-90ca-f9f423c02d38 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.884012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209505ab-72f8-413e-90ca-f9f423c02d38 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.937807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING keystone.common.rbac_enforcer.enforcer [None req-209505ab-72f8-413e-90ca-f9f423c02d38 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 17:28:28.938255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209505ab-72f8-413e-90ca-f9f423c02d38 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.950303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 2/4] 10.222.0.76 () {58 vars in 1324 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:28:28.959097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.974824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f16826-33fa-4b4c-a6c0-d8b87d0420ed None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.980036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496e1be-5105-4670-ae8e-b16ad5d444a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.981177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496e1be-5105-4670-ae8e-b16ad5d444a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.981460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496e1be-5105-4670-ae8e-b16ad5d444a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.981733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496e1be-5105-4670-ae8e-b16ad5d444a2 admin admin] PATH_INFO: `/v3/projects/74aebe409e204672bb76c767e1d15f99` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.983408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496e1be-5105-4670-ae8e-b16ad5d444a2 admin admin] RBAC: Authorizing `identity:get_project(project_id=74aebe409e204672bb76c767e1d15f99)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:29.038003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e496e1be-5105-4670-ae8e-b16ad5d444a2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 17:28:29.038342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496e1be-5105-4670-ae8e-b16ad5d444a2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:29.043680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 3/5] 10.222.0.76 () {56 vars in 1207 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/projects/74aebe409e204672bb76c767e1d15f99 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:31.018430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea825-db97-4206-937b-5a6728aaabab None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.018984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea825-db97-4206-937b-5a6728aaabab None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.018984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea825-db97-4206-937b-5a6728aaabab None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.023106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 3/6] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:31.029645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabdc0b-89c6-44e2-83f9-92ee0f560f48 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.030274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabdc0b-89c6-44e2-83f9-92ee0f560f48 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.030274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabdc0b-89c6-44e2-83f9-92ee0f560f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.051309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0eabdc0b-89c6-44e2-83f9-92ee0f560f48 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:31.052892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0eabdc0b-89c6-44e2-83f9-92ee0f560f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:31.061782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 4/7] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 17:28:31.333880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ea825-db97-4206-937b-5a6728aaabab None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.346609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ea825-db97-4206-937b-5a6728aaabab None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.350840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f110c1-e278-47ea-809a-30d39e2862a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.351942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f110c1-e278-47ea-809a-30d39e2862a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.352320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f110c1-e278-47ea-809a-30d39e2862a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.352593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f110c1-e278-47ea-809a-30d39e2862a1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.360941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f110c1-e278-47ea-809a-30d39e2862a1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:31.362765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f110c1-e278-47ea-809a-30d39e2862a1 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:31.372441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 4/8] 10.222.0.76 () {58 vars in 1324 bytes} [Wed Aug 24 17:28:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:28:31.378319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabdc0b-89c6-44e2-83f9-92ee0f560f48 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.388007 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabdc0b-89c6-44e2-83f9-92ee0f560f48 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.392450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f74a3-7029-4a63-8708-b52910df79a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.393090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f74a3-7029-4a63-8708-b52910df79a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.393375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f74a3-7029-4a63-8708-b52910df79a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.393653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f74a3-7029-4a63-8708-b52910df79a9 admin admin] PATH_INFO: `/v3/users/9c52ecdd884449e5b0fb195998b79885` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.395083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f74a3-7029-4a63-8708-b52910df79a9 admin admin] RBAC: Authorizing `identity:get_user(user_id=9c52ecdd884449e5b0fb195998b79885)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:31.396931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f74a3-7029-4a63-8708-b52910df79a9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:31.398621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 5/9] 10.222.0.76 () {56 vars in 1198 bytes} [Wed Aug 24 17:28:31 2022] GET /identity/v3/users/9c52ecdd884449e5b0fb195998b79885 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:33.449698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194cf77-7b4c-48a0-a48e-be8845ce86b9 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.450288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194cf77-7b4c-48a0-a48e-be8845ce86b9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.450288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194cf77-7b4c-48a0-a48e-be8845ce86b9 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.451617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 5/10] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:33.457041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28e4d2-4c14-48ab-9929-643e4ab1781b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.457509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28e4d2-4c14-48ab-9929-643e4ab1781b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.457611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28e4d2-4c14-48ab-9929-643e4ab1781b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.476073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-bd28e4d2-4c14-48ab-9929-643e4ab1781b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:33.477844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bd28e4d2-4c14-48ab-9929-643e4ab1781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:33.482415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68c34-4c6f-4676-a7ea-f983edb5159c None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.482693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68c34-4c6f-4676-a7ea-f983edb5159c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.482962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68c34-4c6f-4676-a7ea-f983edb5159c None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.484275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 6/11] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:33.489563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.489880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.490155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.532311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG passlib.handlers.bcrypt [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81886) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 17:28:33.532726 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG passlib.handlers.bcrypt [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81886) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 17:28:33.562512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 6/12] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:33.579310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:33.580932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:33.686615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 7/13] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:28:33.836962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28e4d2-4c14-48ab-9929-643e4ab1781b None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.861517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:33.949833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28e4d2-4c14-48ab-9929-643e4ab1781b None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.955263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.987562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040df73-0a5e-4e9e-b05d-c4b0c176ab1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.988204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040df73-0a5e-4e9e-b05d-c4b0c176ab1a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.988482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040df73-0a5e-4e9e-b05d-c4b0c176ab1a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.988759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040df73-0a5e-4e9e-b05d-c4b0c176ab1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.003009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040df73-0a5e-4e9e-b05d-c4b0c176ab1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-4040df73-0a5e-4e9e-b05d-c4b0c176ab1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:34.012031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:34.012990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:28:34.013827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 7/14] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:34.020644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040df73-0a5e-4e9e-b05d-c4b0c176ab1a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.066568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19335563-b692-46d2-ab3f-59f580acfd04 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.069446 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.108536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4a19a-5867-4999-8c4f-d6164f0da87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.108536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a19a-5867-4999-8c4f-d6164f0da87f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.108536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a19a-5867-4999-8c4f-d6164f0da87f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.108536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a19a-5867-4999-8c4f-d6164f0da87f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.114731 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.138791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040df73-0a5e-4e9e-b05d-c4b0c176ab1a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.179342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.180047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.180332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.180608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.181317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:34.181626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.183104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.191122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4a19a-5867-4999-8c4f-d6164f0da87f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.192999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4a19a-5867-4999-8c4f-d6164f0da87f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.197645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 8/15] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:34.203562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.299014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 8/16] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:28:34.304579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4a19a-5867-4999-8c4f-d6164f0da87f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.307437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.310433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29777e7-ed66-4dd4-b47b-fbc7e8f625e1 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.347138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3a944-e4e0-49fd-aca1-d0eee4839524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.347799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a944-e4e0-49fd-aca1-d0eee4839524 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.348084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a944-e4e0-49fd-aca1-d0eee4839524 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.348367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a944-e4e0-49fd-aca1-d0eee4839524 None admin] PATH_INFO: `/v3/users/9c52ecdd884449e5b0fb195998b79885` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.349552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3a944-e4e0-49fd-aca1-d0eee4839524 None admin] RBAC: Authorizing `identity:get_user(user_id=9c52ecdd884449e5b0fb195998b79885)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.351064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3a944-e4e0-49fd-aca1-d0eee4839524 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.353272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 9/17] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/users/9c52ecdd884449e5b0fb195998b79885 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:34.359217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3a944-e4e0-49fd-aca1-d0eee4839524 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.374718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.408201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4a19a-5867-4999-8c4f-d6164f0da87f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.444907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.445832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.446140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.446415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.464751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3a944-e4e0-49fd-aca1-d0eee4839524 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.465459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.466997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:34.485720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:34.486831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:34.487926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:34.487926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:34.487926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:28:34.487926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 9/18] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:34.493286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.502693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a196d53-d882-4a60-ad39-aa8269e89bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.503345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a196d53-d882-4a60-ad39-aa8269e89bff None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.503629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a196d53-d882-4a60-ad39-aa8269e89bff None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.503906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a196d53-d882-4a60-ad39-aa8269e89bff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.505053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a196d53-d882-4a60-ad39-aa8269e89bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.506569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a196d53-d882-4a60-ad39-aa8269e89bff None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.508341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 10/19] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:34.514145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a196d53-d882-4a60-ad39-aa8269e89bff None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.553364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.601050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.602451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11763317-8e3d-4426-9889-66d87088bc15 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.622746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a196d53-d882-4a60-ad39-aa8269e89bff None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.637939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19ddc1-d71d-4b7c-ac32-1e9fd1f2042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.638596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ddc1-d71d-4b7c-ac32-1e9fd1f2042f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.638891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ddc1-d71d-4b7c-ac32-1e9fd1f2042f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.639168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ddc1-d71d-4b7c-ac32-1e9fd1f2042f None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.639827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19ddc1-d71d-4b7c-ac32-1e9fd1f2042f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:34.640151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19ddc1-d71d-4b7c-ac32-1e9fd1f2042f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.641540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19ddc1-d71d-4b7c-ac32-1e9fd1f2042f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.660579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bceea1-6303-4787-9866-353cfb600c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.661220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 10/20] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:34.661400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bceea1-6303-4787-9866-353cfb600c5e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.661600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bceea1-6303-4787-9866-353cfb600c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.661698 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bceea1-6303-4787-9866-353cfb600c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.662491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bceea1-6303-4787-9866-353cfb600c5e None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.domain.id=default, user.id=9c52ecdd884449e5b0fb195998b79885) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:34.662784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bceea1-6303-4787-9866-353cfb600c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.664822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bceea1-6303-4787-9866-353cfb600c5e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.672647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 11/21] 10.222.0.76 () {56 vars in 1322 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/role_assignments?user.id=9c52ecdd884449e5b0fb195998b79885&scope.domain.id=default&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:36.739351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703dd9e-b834-4c1b-98b8-3eda71bdc7b3 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.739864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703dd9e-b834-4c1b-98b8-3eda71bdc7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.739939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703dd9e-b834-4c1b-98b8-3eda71bdc7b3 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.741292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 11/22] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:36.752426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee40697-f6f7-4581-a018-37d1ea1815be None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.753020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee40697-f6f7-4581-a018-37d1ea1815be None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.753020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee40697-f6f7-4581-a018-37d1ea1815be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.782995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-dee40697-f6f7-4581-a018-37d1ea1815be None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:36.784235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-dee40697-f6f7-4581-a018-37d1ea1815be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:36.877606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 12/23] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:37.179934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2c7c0-3271-45d3-81f5-825f78974488 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.180133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2c7c0-3271-45d3-81f5-825f78974488 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.180563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2c7c0-3271-45d3-81f5-825f78974488 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.182182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 12/24] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:37.200861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078d1ac-b2a5-419f-bfa1-6b6b1a3a1bf9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.201368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078d1ac-b2a5-419f-bfa1-6b6b1a3a1bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.201871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078d1ac-b2a5-419f-bfa1-6b6b1a3a1bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.232986 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d078d1ac-b2a5-419f-bfa1-6b6b1a3a1bf9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:37.234201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d078d1ac-b2a5-419f-bfa1-6b6b1a3a1bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:37.308074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2c7c0-3271-45d3-81f5-825f78974488 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.342475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 13/25] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:37.463428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a18f0-3763-44ac-8ada-dfd9d462670b None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.463941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a18f0-3763-44ac-8ada-dfd9d462670b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.464360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a18f0-3763-44ac-8ada-dfd9d462670b None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.466242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 14/26] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:37.480874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9507ed3-df0f-4b81-bef3-0eae54392626 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.481370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9507ed3-df0f-4b81-bef3-0eae54392626 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.481833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9507ed3-df0f-4b81-bef3-0eae54392626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.499620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.504552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2c7c0-3271-45d3-81f5-825f78974488 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.526230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f9507ed3-df0f-4b81-bef3-0eae54392626 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:37.529564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f9507ed3-df0f-4b81-bef3-0eae54392626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:37.546482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.582212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902594bc-2641-4288-ac36-584e1ca23394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.583128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902594bc-2641-4288-ac36-584e1ca23394 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.584420 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902594bc-2641-4288-ac36-584e1ca23394 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.585103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902594bc-2641-4288-ac36-584e1ca23394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.612499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902594bc-2641-4288-ac36-584e1ca23394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-902594bc-2641-4288-ac36-584e1ca23394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:37.619456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:37.620581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:28:37.621487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 13/27] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:37.627066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b6063-2d30-4c47-9678-edf7ebf954b6 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.627378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b6063-2d30-4c47-9678-edf7ebf954b6 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.627714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b6063-2d30-4c47-9678-edf7ebf954b6 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.629553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 14/28] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:37.631632 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b6063-2d30-4c47-9678-edf7ebf954b6 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.693825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 15/29] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:28:37.693825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e44f-349d-44ab-92ff-8eec6a5af0fc None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.694157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e44f-349d-44ab-92ff-8eec6a5af0fc None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.694383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e44f-349d-44ab-92ff-8eec6a5af0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.716873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-5563e44f-349d-44ab-92ff-8eec6a5af0fc None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:37.718897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5563e44f-349d-44ab-92ff-8eec6a5af0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:37.823788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b6063-2d30-4c47-9678-edf7ebf954b6 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.848005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 16/30] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:37.851826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e44f-349d-44ab-92ff-8eec6a5af0fc None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.892096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.892910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.893208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.893480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.895469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:37.895785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.898236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.930844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 15/31] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:37.947374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.015338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.016671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.081518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ffa8b-a7df-4552-beec-0abac9ca3544 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.121197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1d550-2276-42a3-a33e-2eded86086cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.121868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1d550-2276-42a3-a33e-2eded86086cf None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.122165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1d550-2276-42a3-a33e-2eded86086cf None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.122447 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1d550-2276-42a3-a33e-2eded86086cf None admin] PATH_INFO: `/v3/users/9c52ecdd884449e5b0fb195998b79885` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.124132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1d550-2276-42a3-a33e-2eded86086cf None admin] RBAC: Authorizing `identity:get_user(user_id=9c52ecdd884449e5b0fb195998b79885)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.125672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1d550-2276-42a3-a33e-2eded86086cf None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.127719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 16/32] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/users/9c52ecdd884449e5b0fb195998b79885 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:38.128253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e44f-349d-44ab-92ff-8eec6a5af0fc None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.135020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1d550-2276-42a3-a33e-2eded86086cf None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.223548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.224273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.224563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.224843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.246314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.247807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.250746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1d550-2276-42a3-a33e-2eded86086cf None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.266323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:38.268361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:38.269192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:38.270043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:38.270043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:38.270043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:28:38.270043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 17/33] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:38.279495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.288472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1cd23-681d-46fd-9349-7e7e6fc836b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.292549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1cd23-681d-46fd-9349-7e7e6fc836b6 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.292549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1cd23-681d-46fd-9349-7e7e6fc836b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.292549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1cd23-681d-46fd-9349-7e7e6fc836b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.292549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1cd23-681d-46fd-9349-7e7e6fc836b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.292817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1cd23-681d-46fd-9349-7e7e6fc836b6 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.294693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 17/34] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:38.300553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1cd23-681d-46fd-9349-7e7e6fc836b6 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.465126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39735c-1ea9-4e54-9f41-564b5d88f39f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.486266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.508813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.509433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.510219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.510513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.511190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:38.511499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.513005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.534372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 18/35] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:38.535155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.547883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1cd23-681d-46fd-9349-7e7e6fc836b6 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.640079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e5b3c-f03b-4c59-a191-0b12e8437a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.640863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e5b3c-f03b-4c59-a191-0b12e8437a20 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.641083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e5b3c-f03b-4c59-a191-0b12e8437a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.641419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e5b3c-f03b-4c59-a191-0b12e8437a20 None admin] PATH_INFO: `/v3/domains/default/users/9c52ecdd884449e5b0fb195998b79885/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.647542 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e5b3c-f03b-4c59-a191-0b12e8437a20 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9c52ecdd884449e5b0fb195998b79885, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.649199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e5b3c-f03b-4c59-a191-0b12e8437a20 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.658709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca569d-b7ea-44c2-b61f-7e796fcdaf0d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.696187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb67d2b-659e-414f-b1a8-e530ed46b4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.696796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb67d2b-659e-414f-b1a8-e530ed46b4f4 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.697080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb67d2b-659e-414f-b1a8-e530ed46b4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.697354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb67d2b-659e-414f-b1a8-e530ed46b4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.698060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb67d2b-659e-414f-b1a8-e530ed46b4f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.699445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb67d2b-659e-414f-b1a8-e530ed46b4f4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.907795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 18/36] 10.222.0.76 () {58 vars in 1361 bytes} [Wed Aug 24 17:28:38 2022] PUT /identity/v3/domains/default/users/9c52ecdd884449e5b0fb195998b79885/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:38.917392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e5b3c-f03b-4c59-a191-0b12e8437a20 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.950497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 19/37] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 17:28:38 2022] POST /identity/v3/roles => generated 225 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:38.951664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb67d2b-659e-414f-b1a8-e530ed46b4f4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.983087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:39.009718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:39.081780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e5b3c-f03b-4c59-a191-0b12e8437a20 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:39.095273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb67d2b-659e-414f-b1a8-e530ed46b4f4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:39.125075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d81a9-ab9c-4765-8228-dc32ec3d4682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:39.125717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d81a9-ab9c-4765-8228-dc32ec3d4682 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.126047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d81a9-ab9c-4765-8228-dc32ec3d4682 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.126386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d81a9-ab9c-4765-8228-dc32ec3d4682 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.127315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d81a9-ab9c-4765-8228-dc32ec3d4682 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:39.128735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d81a9-ab9c-4765-8228-dc32ec3d4682 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:39.136329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a9c8e-e9bd-4fa2-8553-ddc46d08147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:39.136981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a9c8e-e9bd-4fa2-8553-ddc46d08147c None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.137273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a9c8e-e9bd-4fa2-8553-ddc46d08147c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.137559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a9c8e-e9bd-4fa2-8553-ddc46d08147c None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.138291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a9c8e-e9bd-4fa2-8553-ddc46d08147c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:39.141963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a9c8e-e9bd-4fa2-8553-ddc46d08147c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:39.229903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 20/38] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 17:28:38 2022] POST /identity/v3/roles => generated 219 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:39.231498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51e1a-b48f-40d3-81a2-18104c10f157 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.231787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51e1a-b48f-40d3-81a2-18104c10f157 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.232062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51e1a-b48f-40d3-81a2-18104c10f157 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.233333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 21/39] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:39.234474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243c58-8e69-4672-8db3-ac0517ba473e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.234755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243c58-8e69-4672-8db3-ac0517ba473e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.235028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243c58-8e69-4672-8db3-ac0517ba473e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.236375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 22/40] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:39.242548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f46078-2a0c-4f0c-b341-a8f7d5689545 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.242878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f46078-2a0c-4f0c-b341-a8f7d5689545 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.243274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f46078-2a0c-4f0c-b341-a8f7d5689545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.251861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 19/41] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:38 2022] POST /identity/v3/projects => generated 301 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:39.253233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22495b52-dc05-4b18-96f9-3c4bb78bc27b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.253524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22495b52-dc05-4b18-96f9-3c4bb78bc27b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.253804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22495b52-dc05-4b18-96f9-3c4bb78bc27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.299211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-22495b52-dc05-4b18-96f9-3c4bb78bc27b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:39.300873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-22495b52-dc05-4b18-96f9-3c4bb78bc27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:39.388948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-55f46078-2a0c-4f0c-b341-a8f7d5689545 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:39.391070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-55f46078-2a0c-4f0c-b341-a8f7d5689545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:39.488838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 20/42] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:39.512597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 23/43] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:39.767860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22495b52-dc05-4b18-96f9-3c4bb78bc27b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:39.786423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:39.790651 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f46078-2a0c-4f0c-b341-a8f7d5689545 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:39.825924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:39.874409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22495b52-dc05-4b18-96f9-3c4bb78bc27b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:39.911390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9da37-e2e7-48ad-add7-a837520065ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:39.912094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9da37-e2e7-48ad-add7-a837520065ab None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.912779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9da37-e2e7-48ad-add7-a837520065ab None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.913059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9da37-e2e7-48ad-add7-a837520065ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.914913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9da37-e2e7-48ad-add7-a837520065ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:39.916398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9da37-e2e7-48ad-add7-a837520065ab None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:39.918140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 21/44] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:39.923407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9da37-e2e7-48ad-add7-a837520065ab None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:39.924602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f46078-2a0c-4f0c-b341-a8f7d5689545 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:39.963067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad0650-3325-48c0-b371-e2564a4eb16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:39.963716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad0650-3325-48c0-b371-e2564a4eb16c None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.963957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad0650-3325-48c0-b371-e2564a4eb16c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.964218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad0650-3325-48c0-b371-e2564a4eb16c None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.964898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad0650-3325-48c0-b371-e2564a4eb16c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:39.966315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad0650-3325-48c0-b371-e2564a4eb16c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:40.017823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:40.037736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9da37-e2e7-48ad-add7-a837520065ab None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:40.070261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393d33a-86e7-4db5-a80b-92b018c2169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:40.070891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393d33a-86e7-4db5-a80b-92b018c2169a None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.071177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393d33a-86e7-4db5-a80b-92b018c2169a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.071455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393d33a-86e7-4db5-a80b-92b018c2169a None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.072709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393d33a-86e7-4db5-a80b-92b018c2169a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:40.074484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393d33a-86e7-4db5-a80b-92b018c2169a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:40.076382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 24/45] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 17:28:39 2022] POST /identity/v3/roles => generated 223 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:40.181879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 22/46] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:39 2022] POST /identity/v3/projects => generated 312 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:40.936448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276993e0-d387-41e3-8774-51246f2bbca2 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.936448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276993e0-d387-41e3-8774-51246f2bbca2 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.936448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276993e0-d387-41e3-8774-51246f2bbca2 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.938455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 25/47] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:40.944682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9d8e-4a5d-4fc7-85e0-53f90e2fc6f2 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.945817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9d8e-4a5d-4fc7-85e0-53f90e2fc6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.945954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9d8e-4a5d-4fc7-85e0-53f90e2fc6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.964008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-e20c9d8e-4a5d-4fc7-85e0-53f90e2fc6f2 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:40.965272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e20c9d8e-4a5d-4fc7-85e0-53f90e2fc6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:41.035013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 23/48] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:41.309257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276993e0-d387-41e3-8774-51246f2bbca2 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.400084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.416087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276993e0-d387-41e3-8774-51246f2bbca2 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.451976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863a8cb-fb10-476b-8ea8-a7181cd3e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.452615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863a8cb-fb10-476b-8ea8-a7181cd3e900 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.452899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863a8cb-fb10-476b-8ea8-a7181cd3e900 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.453176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863a8cb-fb10-476b-8ea8-a7181cd3e900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.463168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863a8cb-fb10-476b-8ea8-a7181cd3e900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-2863a8cb-fb10-476b-8ea8-a7181cd3e900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:41.467917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:41.468752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:28:41.469482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 26/49] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:41.474939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c9d8e-4a5d-4fc7-85e0-53f90e2fc6f2 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.544409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.579618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c9d8e-4a5d-4fc7-85e0-53f90e2fc6f2 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.615242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.615913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.616201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.616481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.617831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:41.618222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.619623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.628877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 24/50] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.634966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863a8cb-fb10-476b-8ea8-a7181cd3e900 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.739292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863a8cb-fb10-476b-8ea8-a7181cd3e900 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.775888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e305955-ea75-441b-b455-971e5a8d84c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.776562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e305955-ea75-441b-b455-971e5a8d84c8 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.776844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e305955-ea75-441b-b455-971e5a8d84c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.777120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e305955-ea75-441b-b455-971e5a8d84c8 None admin] PATH_INFO: `/v3/users/9c52ecdd884449e5b0fb195998b79885` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.778287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e305955-ea75-441b-b455-971e5a8d84c8 None admin] RBAC: Authorizing `identity:get_user(user_id=9c52ecdd884449e5b0fb195998b79885)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.779736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e305955-ea75-441b-b455-971e5a8d84c8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.781631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 27/51] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/users/9c52ecdd884449e5b0fb195998b79885 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.787156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.889143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b0ae3-3508-49f9-8a9f-fea3cff0a63b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.920292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.930002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ee0f2-a99c-4d2e-9f4a-99ce2e1b283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.930676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ee0f2-a99c-4d2e-9f4a-99ce2e1b283e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.930966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ee0f2-a99c-4d2e-9f4a-99ce2e1b283e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.931348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ee0f2-a99c-4d2e-9f4a-99ce2e1b283e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.932638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ee0f2-a99c-4d2e-9f4a-99ce2e1b283e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.934205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ee0f2-a99c-4d2e-9f4a-99ce2e1b283e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.936062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 25/52] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.942569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e305955-ea75-441b-b455-971e5a8d84c8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.960519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:42.055986 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e305955-ea75-441b-b455-971e5a8d84c8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.094407 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b41c8-c779-41c2-bd9f-020e2662cc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.095036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b41c8-c779-41c2-bd9f-020e2662cc46 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.095323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b41c8-c779-41c2-bd9f-020e2662cc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.095604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b41c8-c779-41c2-bd9f-020e2662cc46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.096389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b41c8-c779-41c2-bd9f-020e2662cc46 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.domain.id=default, user.id=9c52ecdd884449e5b0fb195998b79885) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:42.096701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b41c8-c779-41c2-bd9f-020e2662cc46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.098242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b41c8-c779-41c2-bd9f-020e2662cc46 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.105353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 28/53] 10.222.0.76 () {56 vars in 1322 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/role_assignments?user.id=9c52ecdd884449e5b0fb195998b79885&scope.domain.id=default&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 530 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:42.146973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a161cd-506b-4158-b11c-13ed3e532093 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.147472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a161cd-506b-4158-b11c-13ed3e532093 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.147568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a161cd-506b-4158-b11c-13ed3e532093 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.149568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 26/54] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:42.155741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da625eb-6d58-45d6-abfd-63e2dc4b3178 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.156010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da625eb-6d58-45d6-abfd-63e2dc4b3178 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.156765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da625eb-6d58-45d6-abfd-63e2dc4b3178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.176586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-3da625eb-6d58-45d6-abfd-63e2dc4b3178 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:42.178034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3da625eb-6d58-45d6-abfd-63e2dc4b3178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:42.271058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 29/55] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:42.542213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a161cd-506b-4158-b11c-13ed3e532093 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:42.646282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a161cd-506b-4158-b11c-13ed3e532093 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.680485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dbe26-30b7-4d32-84b0-cbb284b14e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.681101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dbe26-30b7-4d32-84b0-cbb284b14e9f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.681382 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dbe26-30b7-4d32-84b0-cbb284b14e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.681654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dbe26-30b7-4d32-84b0-cbb284b14e9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.682791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26dbe26-30b7-4d32-84b0-cbb284b14e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.684454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26dbe26-30b7-4d32-84b0-cbb284b14e9f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.686025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 27/56] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:42.692004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da625eb-6d58-45d6-abfd-63e2dc4b3178 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:42.702265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:42.792669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da625eb-6d58-45d6-abfd-63e2dc4b3178 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.833136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6167b-5356-4b72-9536-19944250cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.833136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6167b-5356-4b72-9536-19944250cd7f None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.833136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6167b-5356-4b72-9536-19944250cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.833136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6167b-5356-4b72-9536-19944250cd7f None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.833635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6167b-5356-4b72-9536-19944250cd7f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.833635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6167b-5356-4b72-9536-19944250cd7f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.914897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 30/57] 10.222.0.76 () {60 vars in 1140 bytes} [Wed Aug 24 17:28:42 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:44.846536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ebd65-dc15-410e-980a-bd75dbc8fe58 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.847063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ebd65-dc15-410e-980a-bd75dbc8fe58 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.847151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ebd65-dc15-410e-980a-bd75dbc8fe58 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.848449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 28/58] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:44.854477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb036c2-c0d0-4d31-9e53-0c99867c6a57 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.854886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb036c2-c0d0-4d31-9e53-0c99867c6a57 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.855034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb036c2-c0d0-4d31-9e53-0c99867c6a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.874034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-acb036c2-c0d0-4d31-9e53-0c99867c6a57 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:44.875224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-acb036c2-c0d0-4d31-9e53-0c99867c6a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:44.932110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:44.943443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 31/59] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:45.216132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ebd65-dc15-410e-980a-bd75dbc8fe58 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:45.279217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:45.426363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ebd65-dc15-410e-980a-bd75dbc8fe58 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:45.461758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cd4d4-ea56-4533-a48b-0844bb3dc0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:45.462418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cd4d4-ea56-4533-a48b-0844bb3dc0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:45.462705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cd4d4-ea56-4533-a48b-0844bb3dc0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:45.462985 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cd4d4-ea56-4533-a48b-0844bb3dc0f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:45.464112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cd4d4-ea56-4533-a48b-0844bb3dc0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:45.465562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cd4d4-ea56-4533-a48b-0844bb3dc0f0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:45.467228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 29/60] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:45.472946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb036c2-c0d0-4d31-9e53-0c99867c6a57 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:45.577809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb036c2-c0d0-4d31-9e53-0c99867c6a57 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:45.612496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa16f74-5ec4-452d-8a11-0d78238454c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:45.613109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa16f74-5ec4-452d-8a11-0d78238454c1 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:45.613396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa16f74-5ec4-452d-8a11-0d78238454c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:45.613673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa16f74-5ec4-452d-8a11-0d78238454c1 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:45.614347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa16f74-5ec4-452d-8a11-0d78238454c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:45.615835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa16f74-5ec4-452d-8a11-0d78238454c1 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:45.788012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 32/61] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:28:45 2022] POST /identity/v3/users => generated 275 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:48.849345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2da975-0c09-4486-a393-dd8bd794e7fb None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:48.850363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2da975-0c09-4486-a393-dd8bd794e7fb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:48.850649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2da975-0c09-4486-a393-dd8bd794e7fb None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:48.852300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 30/62] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:48.858271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31ef33-956c-4fe5-8d0c-5f0d95000dde None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:48.858708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31ef33-956c-4fe5-8d0c-5f0d95000dde None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:48.858888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31ef33-956c-4fe5-8d0c-5f0d95000dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:48.884164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-de31ef33-956c-4fe5-8d0c-5f0d95000dde None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:48.885829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-de31ef33-956c-4fe5-8d0c-5f0d95000dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:48.946326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:48.984432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 33/63] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:49.056645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b373a49-115b-4c1f-aa44-38eb4e11cdd4 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.057576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b373a49-115b-4c1f-aa44-38eb4e11cdd4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.058108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b373a49-115b-4c1f-aa44-38eb4e11cdd4 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.059775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 31/64] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:49.066227 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808a9a-54e1-49aa-a434-9c67c637b753 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.066518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808a9a-54e1-49aa-a434-9c67c637b753 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.066962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808a9a-54e1-49aa-a434-9c67c637b753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.091950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-98808a9a-54e1-49aa-a434-9c67c637b753 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:49.093439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-98808a9a-54e1-49aa-a434-9c67c637b753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:49.189129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 34/65] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:49.219567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99483b-159d-4c6f-bbc3-a9cf09c52eda None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.220009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99483b-159d-4c6f-bbc3-a9cf09c52eda None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.220383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99483b-159d-4c6f-bbc3-a9cf09c52eda None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.221915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 32/66] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:49.227441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284157c4-dcc8-470e-a818-f2727dfbf664 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.227913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284157c4-dcc8-470e-a818-f2727dfbf664 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.228894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284157c4-dcc8-470e-a818-f2727dfbf664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.251137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-284157c4-dcc8-470e-a818-f2727dfbf664 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:49.252600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-284157c4-dcc8-470e-a818-f2727dfbf664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:49.349470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 35/67] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:49.362660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99483b-159d-4c6f-bbc3-a9cf09c52eda None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:49.371653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fa3f4-f3b4-454b-b49f-b369b55ccf79 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.371945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fa3f4-f3b4-454b-b49f-b369b55ccf79 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.372227 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fa3f4-f3b4-454b-b49f-b369b55ccf79 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.373593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 36/68] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:49.380061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6d413-07bc-4fd6-a7d8-5ead96b29985 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.382289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6d413-07bc-4fd6-a7d8-5ead96b29985 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.382770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6d413-07bc-4fd6-a7d8-5ead96b29985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.413683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-fef6d413-07bc-4fd6-a7d8-5ead96b29985 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:49.417566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-fef6d413-07bc-4fd6-a7d8-5ead96b29985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:49.478765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:49.528082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 37/69] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:49.573756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6d413-07bc-4fd6-a7d8-5ead96b29985 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:49.589889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99483b-159d-4c6f-bbc3-a9cf09c52eda None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:49.659082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:49.676523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095681ca-0f9c-4776-9815-193a81665feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:49.677090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095681ca-0f9c-4776-9815-193a81665feb None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.689441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095681ca-0f9c-4776-9815-193a81665feb None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.689441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095681ca-0f9c-4776-9815-193a81665feb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.689941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095681ca-0f9c-4776-9815-193a81665feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-095681ca-0f9c-4776-9815-193a81665feb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:49.695228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:49.696112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:28:49.696992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 33/70] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:49.698074 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6d413-07bc-4fd6-a7d8-5ead96b29985 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:49.698153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095681ca-0f9c-4776-9815-193a81665feb None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:49.737845 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55582e-0ea0-4077-b5f4-b949f60a1150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:49.738442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55582e-0ea0-4077-b5f4-b949f60a1150 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.738761 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55582e-0ea0-4077-b5f4-b949f60a1150 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.739036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55582e-0ea0-4077-b5f4-b949f60a1150 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.750129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55582e-0ea0-4077-b5f4-b949f60a1150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-7e55582e-0ea0-4077-b5f4-b949f60a1150 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:49.754901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:49.755825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 38/71] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:49.758763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3953-928c-4013-b489-66a4abadb373 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.759491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3953-928c-4013-b489-66a4abadb373 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.759491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3953-928c-4013-b489-66a4abadb373 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.760032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 39/72] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:49.762737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a3953-928c-4013-b489-66a4abadb373 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:49.793922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:49.819294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095681ca-0f9c-4776-9815-193a81665feb None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:49.855968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ea710-abd4-47f7-ab21-5104153aa67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:49.856652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ea710-abd4-47f7-ab21-5104153aa67e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.856898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ea710-abd4-47f7-ab21-5104153aa67e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.857173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ea710-abd4-47f7-ab21-5104153aa67e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.867604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ea710-abd4-47f7-ab21-5104153aa67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-f25ea710-abd4-47f7-ab21-5104153aa67e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:49.872829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:49.873986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 34/73] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:49.875111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ea710-abd4-47f7-ab21-5104153aa67e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:49.878861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a3953-928c-4013-b489-66a4abadb373 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:49.916553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ca1d0-cb23-4510-9cfd-e4a0f041a924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:49.917182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ca1d0-cb23-4510-9cfd-e4a0f041a924 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.917837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ca1d0-cb23-4510-9cfd-e4a0f041a924 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.917837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ca1d0-cb23-4510-9cfd-e4a0f041a924 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.918480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ca1d0-cb23-4510-9cfd-e4a0f041a924 None admin] RBAC: Adding query filter params (name=member) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:49.918792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ca1d0-cb23-4510-9cfd-e4a0f041a924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:49.920208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ca1d0-cb23-4510-9cfd-e4a0f041a924 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:49.931264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 40/74] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:28:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:49.931264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d2d9e-d6f0-4da3-a8bb-c74c8fe3c582 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.931264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d2d9e-d6f0-4da3-a8bb-c74c8fe3c582 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.931264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d2d9e-d6f0-4da3-a8bb-c74c8fe3c582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.937338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:49.949615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6b3d2d9e-d6f0-4da3-a8bb-c74c8fe3c582 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:49.950886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6b3d2d9e-d6f0-4da3-a8bb-c74c8fe3c582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:49.991893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ea710-abd4-47f7-ab21-5104153aa67e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.021006 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 41/75] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:50.021906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d2d9e-d6f0-4da3-a8bb-c74c8fe3c582 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.043133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.045728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.046844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.047296 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.047740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.048814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:50.049312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.051656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.071727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 35/76] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.071727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.167521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d2d9e-d6f0-4da3-a8bb-c74c8fe3c582 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.197561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6aaf7-85ab-4a3a-85a1-922a2277fbda None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.210726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7fc4-75ad-4a38-ad2c-f480c521a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.211467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7fc4-75ad-4a38-ad2c-f480c521a81b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.211761 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7fc4-75ad-4a38-ad2c-f480c521a81b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.212048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7fc4-75ad-4a38-ad2c-f480c521a81b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.216823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.224537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f7fc4-75ad-4a38-ad2c-f480c521a81b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-508f7fc4-75ad-4a38-ad2c-f480c521a81b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:50.229490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:50.239056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 42/77] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:50.239839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7fc4-75ad-4a38-ad2c-f480c521a81b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.251468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.252445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.252979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.253513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.254799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:50.255314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.257901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.277289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 36/78] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.281978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.334000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.356846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7fc4-75ad-4a38-ad2c-f480c521a81b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.394327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d8487-62a1-452b-8329-f549fc817ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.394968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d8487-62a1-452b-8329-f549fc817ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.395248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d8487-62a1-452b-8329-f549fc817ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.395521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d8487-62a1-452b-8329-f549fc817ba6 None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.397450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc817a93-c38e-4dc9-ad85-3a27f50c9e95 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.410574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d8487-62a1-452b-8329-f549fc817ba6 None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.412057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d8487-62a1-452b-8329-f549fc817ba6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.413748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 43/79] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.414575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d8487-62a1-452b-8329-f549fc817ba6 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.433842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e937f7-7d68-4c5c-a8b3-91111b029b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.434579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e937f7-7d68-4c5c-a8b3-91111b029b31 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.434869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e937f7-7d68-4c5c-a8b3-91111b029b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.435158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e937f7-7d68-4c5c-a8b3-91111b029b31 None admin] PATH_INFO: `/v3/users/9c52ecdd884449e5b0fb195998b79885` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.436350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e937f7-7d68-4c5c-a8b3-91111b029b31 None admin] RBAC: Authorizing `identity:get_user(user_id=9c52ecdd884449e5b0fb195998b79885)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.437871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e937f7-7d68-4c5c-a8b3-91111b029b31 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.439481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 37/80] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/users/9c52ecdd884449e5b0fb195998b79885 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.440257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e937f7-7d68-4c5c-a8b3-91111b029b31 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.488089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.524176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d8487-62a1-452b-8329-f549fc817ba6 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.543567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e937f7-7d68-4c5c-a8b3-91111b029b31 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.561395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.562044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.562329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.562610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.563290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] RBAC: Adding query filter params (name=member) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:50.563598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.564996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.567912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.575762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 44/81] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.581168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.585243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f769a9-4949-47d5-ab78-a00258b92443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.587009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f769a9-4949-47d5-ab78-a00258b92443 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.587009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f769a9-4949-47d5-ab78-a00258b92443 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.587009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f769a9-4949-47d5-ab78-a00258b92443 None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.588449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f769a9-4949-47d5-ab78-a00258b92443 None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.590041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f769a9-4949-47d5-ab78-a00258b92443 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.591551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 38/82] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.592333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f769a9-4949-47d5-ab78-a00258b92443 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.680830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3d577-22a1-4f40-9d85-353d72353140 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.695109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f769a9-4949-47d5-ab78-a00258b92443 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.709627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.719658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddd985-87cf-4be2-aa1c-edf7bad6f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.720288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd985-87cf-4be2-aa1c-edf7bad6f80d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.720569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd985-87cf-4be2-aa1c-edf7bad6f80d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.724114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd985-87cf-4be2-aa1c-edf7bad6f80d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.724114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddd985-87cf-4be2-aa1c-edf7bad6f80d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.724114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddd985-87cf-4be2-aa1c-edf7bad6f80d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.725015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 45/83] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.725780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddd985-87cf-4be2-aa1c-edf7bad6f80d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.731899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf5cf7-85fd-4152-aa42-623687a9b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.732514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf5cf7-85fd-4152-aa42-623687a9b031 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.732794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf5cf7-85fd-4152-aa42-623687a9b031 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.733074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf5cf7-85fd-4152-aa42-623687a9b031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.799436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.803638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf5cf7-85fd-4152-aa42-623687a9b031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.805100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf5cf7-85fd-4152-aa42-623687a9b031 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.836741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddd985-87cf-4be2-aa1c-edf7bad6f80d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.872197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0376aa9-504e-4d6f-980a-889af4414948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.872819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0376aa9-504e-4d6f-980a-889af4414948 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.873105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0376aa9-504e-4d6f-980a-889af4414948 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0376aa9-504e-4d6f-980a-889af4414948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.909779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 39/84] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:50.910677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf5cf7-85fd-4152-aa42-623687a9b031 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.927305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.961166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0376aa9-504e-4d6f-980a-889af4414948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.962766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0376aa9-504e-4d6f-980a-889af4414948 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.024609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf5cf7-85fd-4152-aa42-623687a9b031 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.031964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.061532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17db24-f8d3-4aba-b51d-9c053dc9fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.062146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17db24-f8d3-4aba-b51d-9c053dc9fba7 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.062435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17db24-f8d3-4aba-b51d-9c053dc9fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.062716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17db24-f8d3-4aba-b51d-9c053dc9fba7 None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.063892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17db24-f8d3-4aba-b51d-9c053dc9fba7 None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.065365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17db24-f8d3-4aba-b51d-9c053dc9fba7 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.067308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 40/85] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:51.068088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17db24-f8d3-4aba-b51d-9c053dc9fba7 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.068661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 46/86] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:51.069183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0376aa9-504e-4d6f-980a-889af4414948 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.153463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.172711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0376aa9-504e-4d6f-980a-889af4414948 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.173265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17db24-f8d3-4aba-b51d-9c053dc9fba7 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.207549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21226683-e72b-4677-9cd2-9df3c805b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.208161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226683-e72b-4677-9cd2-9df3c805b3d9 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.208446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226683-e72b-4677-9cd2-9df3c805b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.208636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e673aac-2082-4022-bb41-5dce025f20b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.208753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226683-e72b-4677-9cd2-9df3c805b3d9 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.209242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e673aac-2082-4022-bb41-5dce025f20b4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.209363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21226683-e72b-4677-9cd2-9df3c805b3d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.209523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e673aac-2082-4022-bb41-5dce025f20b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.209847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e673aac-2082-4022-bb41-5dce025f20b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.210878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21226683-e72b-4677-9cd2-9df3c805b3d9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.278266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e673aac-2082-4022-bb41-5dce025f20b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.279759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e673aac-2082-4022-bb41-5dce025f20b4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.297356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 47/87] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:28:51 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:51.298251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21226683-e72b-4677-9cd2-9df3c805b3d9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.330758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.387599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.417916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 41/88] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:51.417916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e673aac-2082-4022-bb41-5dce025f20b4 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.437574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21226683-e72b-4677-9cd2-9df3c805b3d9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.490119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369acd0-e337-4c5d-a25f-a71a1a09ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.490835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369acd0-e337-4c5d-a25f-a71a1a09ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.491067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369acd0-e337-4c5d-a25f-a71a1a09ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.491345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369acd0-e337-4c5d-a25f-a71a1a09ffe6 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.492601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369acd0-e337-4c5d-a25f-a71a1a09ffe6 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.496280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369acd0-e337-4c5d-a25f-a71a1a09ffe6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.496412 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 48/89] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:51.502768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369acd0-e337-4c5d-a25f-a71a1a09ffe6 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.562803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e673aac-2082-4022-bb41-5dce025f20b4 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.621397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc63a2-16ee-4ab9-a6f7-a89624c16ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.622400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc63a2-16ee-4ab9-a6f7-a89624c16ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.622883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc63a2-16ee-4ab9-a6f7-a89624c16ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.623334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc63a2-16ee-4ab9-a6f7-a89624c16ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.628189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.648635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369acd0-e337-4c5d-a25f-a71a1a09ffe6 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.692872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.709670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532ae98-4eae-422c-ab1d-9608a313c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.710367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532ae98-4eae-422c-ab1d-9608a313c725 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.710652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532ae98-4eae-422c-ab1d-9608a313c725 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.710931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532ae98-4eae-422c-ab1d-9608a313c725 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.712095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532ae98-4eae-422c-ab1d-9608a313c725 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.714183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532ae98-4eae-422c-ab1d-9608a313c725 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.715902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 49/90] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:51.716702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532ae98-4eae-422c-ab1d-9608a313c725 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.722964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc63a2-16ee-4ab9-a6f7-a89624c16ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.724995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc63a2-16ee-4ab9-a6f7-a89624c16ab5 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.865704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532ae98-4eae-422c-ab1d-9608a313c725 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.870439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 42/91] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:51.871671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc63a2-16ee-4ab9-a6f7-a89624c16ab5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.919628 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efb17-a2e6-4191-96b9-c831a2fbf345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.920294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efb17-a2e6-4191-96b9-c831a2fbf345 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.920586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efb17-a2e6-4191-96b9-c831a2fbf345 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.920866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efb17-a2e6-4191-96b9-c831a2fbf345 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.922080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3efb17-a2e6-4191-96b9-c831a2fbf345 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.923562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3efb17-a2e6-4191-96b9-c831a2fbf345 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.926191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 50/92] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:51.930085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efb17-a2e6-4191-96b9-c831a2fbf345 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.974641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.977465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.999573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc63a2-16ee-4ab9-a6f7-a89624c16ab5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:52.039223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:52.040264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.041343 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.041816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.042776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] RBAC: Adding query filter params (role.id=fff136f214f2428895daf10483c542d1, scope.project.id=2f9e413c56d445ab804ef9a518cc2202, user.id=1d96bcf28939461487e0fd8a9e516811) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:52.043293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:52.045017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:52.052966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 43/93] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/role_assignments?user.id=1d96bcf28939461487e0fd8a9e516811&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=fff136f214f2428895daf10483c542d1 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:52.054066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:52.153807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efb17-a2e6-4191-96b9-c831a2fbf345 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:52.241303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:52.242352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.244303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.246359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.247168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=2f9e413c56d445ab804ef9a518cc2202, user.id=9c52ecdd884449e5b0fb195998b79885) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:52.247489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:52.248998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:52.257033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 51/94] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/role_assignments?user.id=9c52ecdd884449e5b0fb195998b79885&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:52.257960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:52.268762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1aed7-fdd7-4873-968f-ad0c5fede332 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:52.312556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d04622-b52d-4ef8-9edb-700838ca9bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:52.313686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d04622-b52d-4ef8-9edb-700838ca9bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.314267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d04622-b52d-4ef8-9edb-700838ca9bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.314740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d04622-b52d-4ef8-9edb-700838ca9bf5 None admin] PATH_INFO: `/v3/projects/af6939c5e5844d23a68c27694bdd64c0` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.316594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d04622-b52d-4ef8-9edb-700838ca9bf5 None admin] RBAC: Authorizing `identity:get_project(project_id=af6939c5e5844d23a68c27694bdd64c0)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:52.318419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d04622-b52d-4ef8-9edb-700838ca9bf5 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:52.325616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 44/95] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:52 2022] GET /identity/v3/projects/af6939c5e5844d23a68c27694bdd64c0 => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:52.334904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d04622-b52d-4ef8-9edb-700838ca9bf5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:52.357187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:52.425318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:52.441881 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea44621-5b9e-435d-8691-892f8acf8d08 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:52.500448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d04622-b52d-4ef8-9edb-700838ca9bf5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:52.542630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775161e-12f5-427e-ad8d-29b2e95191c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:52.542630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775161e-12f5-427e-ad8d-29b2e95191c7 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.542630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775161e-12f5-427e-ad8d-29b2e95191c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.542630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775161e-12f5-427e-ad8d-29b2e95191c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.543046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775161e-12f5-427e-ad8d-29b2e95191c7 None admin] RBAC: Adding query filter params (role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=2f9e413c56d445ab804ef9a518cc2202, user.id=1d96bcf28939461487e0fd8a9e516811) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:52.543046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775161e-12f5-427e-ad8d-29b2e95191c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:52.543336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775161e-12f5-427e-ad8d-29b2e95191c7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:52.557558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 52/96] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:52 2022] GET /identity/v3/role_assignments?user.id=1d96bcf28939461487e0fd8a9e516811&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=d6fe1aca91e74aad9907c855501f588f => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:52.566564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395bc17-8c93-4f47-9550-47051947b42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:52.569059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395bc17-8c93-4f47-9550-47051947b42b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.569645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395bc17-8c93-4f47-9550-47051947b42b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.572827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395bc17-8c93-4f47-9550-47051947b42b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.574096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395bc17-8c93-4f47-9550-47051947b42b None admin] RBAC: Adding query filter params (role.id=fff136f214f2428895daf10483c542d1, scope.project.id=af6939c5e5844d23a68c27694bdd64c0, user.id=1d96bcf28939461487e0fd8a9e516811) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:52.574617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395bc17-8c93-4f47-9550-47051947b42b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:52.576327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395bc17-8c93-4f47-9550-47051947b42b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:52.587624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 45/97] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:52 2022] GET /identity/v3/role_assignments?user.id=1d96bcf28939461487e0fd8a9e516811&scope.project.id=af6939c5e5844d23a68c27694bdd64c0&role.id=fff136f214f2428895daf10483c542d1 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:53.963041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ed9c-e31e-4980-b76b-72562de3f013 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.963655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ed9c-e31e-4980-b76b-72562de3f013 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.963655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ed9c-e31e-4980-b76b-72562de3f013 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.965103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 53/98] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:53.971994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0f22-3a67-41de-bd1a-72e5231126dc None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.972467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0f22-3a67-41de-bd1a-72e5231126dc None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.972658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0f22-3a67-41de-bd1a-72e5231126dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.998992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-8c5c0f22-3a67-41de-bd1a-72e5231126dc None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:54.000220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8c5c0f22-3a67-41de-bd1a-72e5231126dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:54.044815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a0dc2-6d27-4aa0-80d5-6002eef1b585 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.045208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a0dc2-6d27-4aa0-80d5-6002eef1b585 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.045402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a0dc2-6d27-4aa0-80d5-6002eef1b585 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.046889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 54/99] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:54.060173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f683c2-8021-497a-8e63-b43415b8e881 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.060464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f683c2-8021-497a-8e63-b43415b8e881 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.060738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f683c2-8021-497a-8e63-b43415b8e881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.090343 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 46/100] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:54.098916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f6f683c2-8021-497a-8e63-b43415b8e881 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:54.100235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f6f683c2-8021-497a-8e63-b43415b8e881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:54.190361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 55/101] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:28:54.498954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c0f22-3a67-41de-bd1a-72e5231126dc None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.508296 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.632618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c0f22-3a67-41de-bd1a-72e5231126dc None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.670895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f683c2-8021-497a-8e63-b43415b8e881 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.674033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48143687-eedb-4aa9-90f4-aa53fddd3d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.675107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48143687-eedb-4aa9-90f4-aa53fddd3d96 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.675586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48143687-eedb-4aa9-90f4-aa53fddd3d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.676081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48143687-eedb-4aa9-90f4-aa53fddd3d96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.701037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48143687-eedb-4aa9-90f4-aa53fddd3d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-48143687-eedb-4aa9-90f4-aa53fddd3d96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:54.707720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:54.708585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:28:54.709339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:28:54.717262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 47/102] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:54.718655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48143687-eedb-4aa9-90f4-aa53fddd3d96 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.861307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.919486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48143687-eedb-4aa9-90f4-aa53fddd3d96 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.928617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.945320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f683c2-8021-497a-8e63-b43415b8e881 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:55.011716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb122-29f8-45e8-aea0-7273ea6c36a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:55.013048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb122-29f8-45e8-aea0-7273ea6c36a3 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.013485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb122-29f8-45e8-aea0-7273ea6c36a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.013931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb122-29f8-45e8-aea0-7273ea6c36a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.015288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cb122-29f8-45e8-aea0-7273ea6c36a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:55.016894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cb122-29f8-45e8-aea0-7273ea6c36a3 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:55.018804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 56/103] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:55.020230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2636ce-7a2c-48b4-860b-ed7d18f9e23c None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.020741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2636ce-7a2c-48b4-860b-ed7d18f9e23c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.021186 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2636ce-7a2c-48b4-860b-ed7d18f9e23c None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.022393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61273f-b7ad-4fa4-8eb3-9ebeb3190b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:55.023124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61273f-b7ad-4fa4-8eb3-9ebeb3190b3b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.023282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 57/104] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:55.026951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61273f-b7ad-4fa4-8eb3-9ebeb3190b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.026951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61273f-b7ad-4fa4-8eb3-9ebeb3190b3b None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.026951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61273f-b7ad-4fa4-8eb3-9ebeb3190b3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:55.026951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61273f-b7ad-4fa4-8eb3-9ebeb3190b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:55.028162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2636ce-7a2c-48b4-860b-ed7d18f9e23c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:55.030612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61273f-b7ad-4fa4-8eb3-9ebeb3190b3b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:55.048494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 48/105] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:55.052231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dacf37-d405-4b84-9a7b-97a281178b13 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.052733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dacf37-d405-4b84-9a7b-97a281178b13 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.053189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dacf37-d405-4b84-9a7b-97a281178b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.083929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-32dacf37-d405-4b84-9a7b-97a281178b13 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:55.085651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-32dacf37-d405-4b84-9a7b-97a281178b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:55.219435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2636ce-7a2c-48b4-860b-ed7d18f9e23c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:55.234252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 49/106] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:55.235148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dacf37-d405-4b84-9a7b-97a281178b13 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:55.291705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3cd07-381d-4dd2-ad70-7dfa7c78e856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:55.292331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cd07-381d-4dd2-ad70-7dfa7c78e856 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.292619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cd07-381d-4dd2-ad70-7dfa7c78e856 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.292897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cd07-381d-4dd2-ad70-7dfa7c78e856 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.293556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3cd07-381d-4dd2-ad70-7dfa7c78e856 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:55.295067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3cd07-381d-4dd2-ad70-7dfa7c78e856 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:55.353608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:55.415166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dacf37-d405-4b84-9a7b-97a281178b13 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:55.416058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 58/107] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:55 2022] POST /identity/v3/projects => generated 302 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:55.443856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e63f3-05c3-45ff-8567-d47ecb364db6 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.444428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e63f3-05c3-45ff-8567-d47ecb364db6 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.444937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e63f3-05c3-45ff-8567-d47ecb364db6 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.447278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 59/108] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:55.455683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33435-9102-4c71-83ea-10b7e69cfb0a None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.456354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33435-9102-4c71-83ea-10b7e69cfb0a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.456832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33435-9102-4c71-83ea-10b7e69cfb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.468322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c89ff-f026-4107-a585-af0e520e4385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:55.470651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c89ff-f026-4107-a585-af0e520e4385 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.470886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c89ff-f026-4107-a585-af0e520e4385 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.471145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c89ff-f026-4107-a585-af0e520e4385 None admin] PATH_INFO: `/v3/users/3eb0c19623d54fd7bc324b91920cf2dc` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.473431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:55.491898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c89ff-f026-4107-a585-af0e520e4385 None admin] RBAC: Authorizing `identity:get_user(user_id=3eb0c19623d54fd7bc324b91920cf2dc)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:55.493388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c89ff-f026-4107-a585-af0e520e4385 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:55.496067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 50/109] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:55 2022] GET /identity/v3/users/3eb0c19623d54fd7bc324b91920cf2dc => generated 289 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:55.497431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-bef33435-9102-4c71-83ea-10b7e69cfb0a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:55.498964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bef33435-9102-4c71-83ea-10b7e69cfb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:55.502187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c89ff-f026-4107-a585-af0e520e4385 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:55.607940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 60/110] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:55.617886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3019-7ffe-4282-ac29-abe8d07cb28c None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.618909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3019-7ffe-4282-ac29-abe8d07cb28c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.619444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3019-7ffe-4282-ac29-abe8d07cb28c None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.621249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 61/111] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:55 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:55.622894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb2ba-7e0c-4281-816f-919500970e89 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.623435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb2ba-7e0c-4281-816f-919500970e89 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.623909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb2ba-7e0c-4281-816f-919500970e89 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.625513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 62/112] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:55.625985 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c89ff-f026-4107-a585-af0e520e4385 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:55.627178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ba50-5cec-4230-adfa-637a04da4e28 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.627805 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ba50-5cec-4230-adfa-637a04da4e28 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.628283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ba50-5cec-4230-adfa-637a04da4e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.646491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:55.667674 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f2a2ba50-5cec-4230-adfa-637a04da4e28 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:55.668448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9cf26-e25f-43bc-b452-35a49531af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:55.671279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9cf26-e25f-43bc-b452-35a49531af59 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.671279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9cf26-e25f-43bc-b452-35a49531af59 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.671279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9cf26-e25f-43bc-b452-35a49531af59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.671643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f2a2ba50-5cec-4230-adfa-637a04da4e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:55.759897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9cf26-e25f-43bc-b452-35a49531af59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:55.761550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9cf26-e25f-43bc-b452-35a49531af59 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:55.765916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 63/113] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:55.768203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84351e-11c2-40cc-b390-0f913724536e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.768726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84351e-11c2-40cc-b390-0f913724536e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.769197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84351e-11c2-40cc-b390-0f913724536e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.798897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6b84351e-11c2-40cc-b390-0f913724536e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:55.800610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6b84351e-11c2-40cc-b390-0f913724536e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:55.887454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 51/114] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:28:55.888840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9cf26-e25f-43bc-b452-35a49531af59 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:55.890271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:55.896624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 64/115] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:55.899040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84351e-11c2-40cc-b390-0f913724536e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:55.941047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.111502 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84351e-11c2-40cc-b390-0f913724536e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.133471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9cf26-e25f-43bc-b452-35a49531af59 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.154165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1875f-cfc7-4180-9545-1002974f45e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.155305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1875f-cfc7-4180-9545-1002974f45e5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.155786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1875f-cfc7-4180-9545-1002974f45e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.156257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1875f-cfc7-4180-9545-1002974f45e5 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.157775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1875f-cfc7-4180-9545-1002974f45e5 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:56.159453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1875f-cfc7-4180-9545-1002974f45e5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:56.161432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 65/116] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:55 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:56.162491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1875f-cfc7-4180-9545-1002974f45e5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.178908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3797515-e02a-48b5-947a-4f5f8c22d987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.180076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3797515-e02a-48b5-947a-4f5f8c22d987 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.180580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3797515-e02a-48b5-947a-4f5f8c22d987 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.181097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3797515-e02a-48b5-947a-4f5f8c22d987 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.201886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3797515-e02a-48b5-947a-4f5f8c22d987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-b3797515-e02a-48b5-947a-4f5f8c22d987 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:56.203308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:56.204221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:28:56.204995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 52/117] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:56.210378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3797515-e02a-48b5-947a-4f5f8c22d987 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.241428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.288710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1875f-cfc7-4180-9545-1002974f45e5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.299646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.325225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3797515-e02a-48b5-947a-4f5f8c22d987 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.331741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117da6d3-07ef-4d6f-9ef1-7f147a231634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.332394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117da6d3-07ef-4d6f-9ef1-7f147a231634 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.332682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117da6d3-07ef-4d6f-9ef1-7f147a231634 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.332958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117da6d3-07ef-4d6f-9ef1-7f147a231634 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.348101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117da6d3-07ef-4d6f-9ef1-7f147a231634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-117da6d3-07ef-4d6f-9ef1-7f147a231634 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:56.354837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:56.356130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:28:56.357105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 66/118] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:56.357909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117da6d3-07ef-4d6f-9ef1-7f147a231634 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.380371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c42625-7743-4c68-b235-d20751b4a639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.380930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c42625-7743-4c68-b235-d20751b4a639 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.381213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c42625-7743-4c68-b235-d20751b4a639 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.381524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c42625-7743-4c68-b235-d20751b4a639 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.393708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c42625-7743-4c68-b235-d20751b4a639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-93c42625-7743-4c68-b235-d20751b4a639 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:56.402416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:56.403321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:28:56.404142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:28:56.404723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 53/119] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:56.406207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c42625-7743-4c68-b235-d20751b4a639 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.507112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117da6d3-07ef-4d6f-9ef1-7f147a231634 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.544004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.559426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.560664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.560664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.560664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.561704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] RBAC: Adding query filter params (role.id=f9e4fcd3595c43609a8c949b1996b02c, scope.project.id=2f9e413c56d445ab804ef9a518cc2202, user.id=3eb0c19623d54fd7bc324b91920cf2dc) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:56.562055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:56.564377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:56.572882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 67/120] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/role_assignments?user.id=3eb0c19623d54fd7bc324b91920cf2dc&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=f9e4fcd3595c43609a8c949b1996b02c => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:56.573671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.575535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c42625-7743-4c68-b235-d20751b4a639 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.658826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.660182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.660623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.661078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.663249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:56.666031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:56.670786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:56.676241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.702342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 54/121] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:56.702342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.746721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2af709-d2f3-48d8-b557-d4cbb99c8cfa None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.802434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0027675-b8a0-4aec-bfc4-515f4304e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.803527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0027675-b8a0-4aec-bfc4-515f4304e489 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.804006 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0027675-b8a0-4aec-bfc4-515f4304e489 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.804473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0027675-b8a0-4aec-bfc4-515f4304e489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.818304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0027675-b8a0-4aec-bfc4-515f4304e489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c0027675-b8a0-4aec-bfc4-515f4304e489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:56.825919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:56.826852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:56.827605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:28:56.830026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 68/122] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:56.831266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0027675-b8a0-4aec-bfc4-515f4304e489 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.844538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c9930-ecef-4d0d-a768-0c766ac0bea9 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.886777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.887769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.888064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.888346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.889041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:56.889351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:56.891599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:56.920748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 55/123] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:56.927773 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.933786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.016752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.045773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0027675-b8a0-4aec-bfc4-515f4304e489 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.081761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff497d8-2af6-4619-98f9-0d8c48059f49 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.105205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.105825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.106134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.106408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.107143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:57.107449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.108867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.118964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 69/124] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.118964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.130958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69791a-8bad-47df-9203-cf0e0ed32545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.131970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69791a-8bad-47df-9203-cf0e0ed32545 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.132510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69791a-8bad-47df-9203-cf0e0ed32545 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.132989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69791a-8bad-47df-9203-cf0e0ed32545 None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.135198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69791a-8bad-47df-9203-cf0e0ed32545 None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.137626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69791a-8bad-47df-9203-cf0e0ed32545 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.139961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 56/125] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.141018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69791a-8bad-47df-9203-cf0e0ed32545 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.244335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaacb5-b883-4554-92d8-6d5228231df1 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.254887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.268617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69791a-8bad-47df-9203-cf0e0ed32545 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.284665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.285304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.285589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.285879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.286618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:57.286924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.288350 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.298990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 70/126] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.303890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.310838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.321806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a3b1-ad2f-42bb-b083-ca2cd98ef7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.322498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a3b1-ad2f-42bb-b083-ca2cd98ef7de None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.322810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a3b1-ad2f-42bb-b083-ca2cd98ef7de None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.323047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a3b1-ad2f-42bb-b083-ca2cd98ef7de None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.325032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085a3b1-ad2f-42bb-b083-ca2cd98ef7de None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.326570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085a3b1-ad2f-42bb-b083-ca2cd98ef7de None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.333470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 57/127] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.333470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a3b1-ad2f-42bb-b083-ca2cd98ef7de None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.456098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21c245-d4ea-45ca-9ada-2d21df4d7542 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.486497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a3b1-ad2f-42bb-b083-ca2cd98ef7de None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.512568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefed504-a673-4685-9786-20b2fb5d3c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.513814 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefed504-a673-4685-9786-20b2fb5d3c2c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.514339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefed504-a673-4685-9786-20b2fb5d3c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.515023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefed504-a673-4685-9786-20b2fb5d3c2c None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.517164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefed504-a673-4685-9786-20b2fb5d3c2c None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.519901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefed504-a673-4685-9786-20b2fb5d3c2c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.525951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 71/128] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.528403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefed504-a673-4685-9786-20b2fb5d3c2c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.534252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b63ed-0d02-41b8-afb3-8c3b52a7806f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.534933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b63ed-0d02-41b8-afb3-8c3b52a7806f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.535241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b63ed-0d02-41b8-afb3-8c3b52a7806f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.535626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b63ed-0d02-41b8-afb3-8c3b52a7806f None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.536917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b63ed-0d02-41b8-afb3-8c3b52a7806f None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.538420 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b63ed-0d02-41b8-afb3-8c3b52a7806f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.540095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 58/129] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.542749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b63ed-0d02-41b8-afb3-8c3b52a7806f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.584595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.615453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.654424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefed504-a673-4685-9786-20b2fb5d3c2c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.661180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b63ed-0d02-41b8-afb3-8c3b52a7806f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.695492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985025-4c7f-4a83-8790-92e19b5a759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.696207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985025-4c7f-4a83-8790-92e19b5a759a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.696496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985025-4c7f-4a83-8790-92e19b5a759a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.697538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985025-4c7f-4a83-8790-92e19b5a759a None admin] PATH_INFO: `/v3/users/9c52ecdd884449e5b0fb195998b79885` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.698965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985025-4c7f-4a83-8790-92e19b5a759a None admin] RBAC: Authorizing `identity:get_user(user_id=9c52ecdd884449e5b0fb195998b79885)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.700456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985025-4c7f-4a83-8790-92e19b5a759a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.706826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 72/130] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/users/9c52ecdd884449e5b0fb195998b79885 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.706826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985025-4c7f-4a83-8790-92e19b5a759a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.708554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0f9da-a4e7-4300-8b0c-e1c8b941cc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.709206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0f9da-a4e7-4300-8b0c-e1c8b941cc46 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.709498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0f9da-a4e7-4300-8b0c-e1c8b941cc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.709776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0f9da-a4e7-4300-8b0c-e1c8b941cc46 None admin] PATH_INFO: `/v3/projects/af6939c5e5844d23a68c27694bdd64c0` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.714696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0f9da-a4e7-4300-8b0c-e1c8b941cc46 None admin] RBAC: Authorizing `identity:get_project(project_id=af6939c5e5844d23a68c27694bdd64c0)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.716503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0f9da-a4e7-4300-8b0c-e1c8b941cc46 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.718918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 59/131] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/projects/af6939c5e5844d23a68c27694bdd64c0 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.720040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0f9da-a4e7-4300-8b0c-e1c8b941cc46 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.833564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985025-4c7f-4a83-8790-92e19b5a759a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.838559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0f9da-a4e7-4300-8b0c-e1c8b941cc46 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.843039 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.869515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.874473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef4495-9790-413d-825c-e376d80b8014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.874813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b70ce-298e-4d96-ac30-e62d1950ba88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.875152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef4495-9790-413d-825c-e376d80b8014 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.875439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef4495-9790-413d-825c-e376d80b8014 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.875531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b70ce-298e-4d96-ac30-e62d1950ba88 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.875720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b70ce-298e-4d96-ac30-e62d1950ba88 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.875811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef4495-9790-413d-825c-e376d80b8014 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202/users/1d96bcf28939461487e0fd8a9e516811/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.875998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b70ce-298e-4d96-ac30-e62d1950ba88 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.877337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b70ce-298e-4d96-ac30-e62d1950ba88 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.877743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef4495-9790-413d-825c-e376d80b8014 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f9e413c56d445ab804ef9a518cc2202, user_id=1d96bcf28939461487e0fd8a9e516811, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.879236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef4495-9790-413d-825c-e376d80b8014 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.880025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b70ce-298e-4d96-ac30-e62d1950ba88 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.882097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 73/132] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:57.888508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b70ce-298e-4d96-ac30-e62d1950ba88 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.948846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 60/133] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:57 2022] PUT /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202/users/1d96bcf28939461487e0fd8a9e516811/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:57.958887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef4495-9790-413d-825c-e376d80b8014 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.004739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b70ce-298e-4d96-ac30-e62d1950ba88 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.051746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21ad33-9a76-4b87-a72f-93d59c87916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.052727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21ad33-9a76-4b87-a72f-93d59c87916d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.053222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21ad33-9a76-4b87-a72f-93d59c87916d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.054086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21ad33-9a76-4b87-a72f-93d59c87916d None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.057811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21ad33-9a76-4b87-a72f-93d59c87916d None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.059836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21ad33-9a76-4b87-a72f-93d59c87916d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.062721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 74/134] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:58.064343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628a21e-7a03-410f-b2e0-b401642e44a9 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.065090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628a21e-7a03-410f-b2e0-b401642e44a9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.065519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628a21e-7a03-410f-b2e0-b401642e44a9 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.067368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 75/135] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:58.068317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628a21e-7a03-410f-b2e0-b401642e44a9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.132295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:58.153896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef4495-9790-413d-825c-e376d80b8014 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.206459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081fa28-d6f2-45fa-b14f-b9d6cd7face9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.207316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628a21e-7a03-410f-b2e0-b401642e44a9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.207479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081fa28-d6f2-45fa-b14f-b9d6cd7face9 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.207935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081fa28-d6f2-45fa-b14f-b9d6cd7face9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.208370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081fa28-d6f2-45fa-b14f-b9d6cd7face9 None admin] PATH_INFO: `/v3/projects/af6939c5e5844d23a68c27694bdd64c0/users/1d96bcf28939461487e0fd8a9e516811/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.213682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081fa28-d6f2-45fa-b14f-b9d6cd7face9 None admin] RBAC: Authorizing `identity:create_grant(project_id=af6939c5e5844d23a68c27694bdd64c0, user_id=1d96bcf28939461487e0fd8a9e516811, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.217612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081fa28-d6f2-45fa-b14f-b9d6cd7face9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.249014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07398d-57fd-4cc4-a66d-5b824931e99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.249688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07398d-57fd-4cc4-a66d-5b824931e99d None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.250078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07398d-57fd-4cc4-a66d-5b824931e99d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.250364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07398d-57fd-4cc4-a66d-5b824931e99d None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202/users/1d96bcf28939461487e0fd8a9e516811/roles/d6fe1aca91e74aad9907c855501f588f` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.252535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07398d-57fd-4cc4-a66d-5b824931e99d None admin] RBAC: Authorizing `identity:create_grant(project_id=2f9e413c56d445ab804ef9a518cc2202, user_id=1d96bcf28939461487e0fd8a9e516811, role_id=d6fe1aca91e74aad9907c855501f588f)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.254164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07398d-57fd-4cc4-a66d-5b824931e99d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.291888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 61/136] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:57 2022] PUT /identity/v3/projects/af6939c5e5844d23a68c27694bdd64c0/users/1d96bcf28939461487e0fd8a9e516811/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:58.294845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081fa28-d6f2-45fa-b14f-b9d6cd7face9 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.365210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:58.367837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 76/137] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:58 2022] PUT /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202/users/1d96bcf28939461487e0fd8a9e516811/roles/d6fe1aca91e74aad9907c855501f588f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:58.369765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b60f5-e2d8-4e90-a7b7-0b99db657c67 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.370255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b60f5-e2d8-4e90-a7b7-0b99db657c67 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.370542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b60f5-e2d8-4e90-a7b7-0b99db657c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.407088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e17b60f5-e2d8-4e90-a7b7-0b99db657c67 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:58.408179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e17b60f5-e2d8-4e90-a7b7-0b99db657c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:58.429141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081fa28-d6f2-45fa-b14f-b9d6cd7face9 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.472351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db382409-c472-4332-9b09-a826daf4c7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.473384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db382409-c472-4332-9b09-a826daf4c7fd None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.473878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db382409-c472-4332-9b09-a826daf4c7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.474349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db382409-c472-4332-9b09-a826daf4c7fd None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202/users/9c52ecdd884449e5b0fb195998b79885/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.476799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db382409-c472-4332-9b09-a826daf4c7fd None admin] RBAC: Authorizing `identity:create_grant(project_id=2f9e413c56d445ab804ef9a518cc2202, user_id=9c52ecdd884449e5b0fb195998b79885, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.479168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db382409-c472-4332-9b09-a826daf4c7fd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.560335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 62/138] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:58 2022] PUT /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202/users/9c52ecdd884449e5b0fb195998b79885/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:58.562369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 77/139] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:58.641627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ee690-7c66-44b2-956d-2183711b4675 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.641967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ee690-7c66-44b2-956d-2183711b4675 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.642293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ee690-7c66-44b2-956d-2183711b4675 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.643841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 63/140] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:58.652404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cecb7f-1d9e-4ebf-b5ed-a52f3cdf4958 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.652718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cecb7f-1d9e-4ebf-b5ed-a52f3cdf4958 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.652968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cecb7f-1d9e-4ebf-b5ed-a52f3cdf4958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.675293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-20cecb7f-1d9e-4ebf-b5ed-a52f3cdf4958 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:58.676452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-20cecb7f-1d9e-4ebf-b5ed-a52f3cdf4958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:58.710835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:58.764303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 78/141] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:58.927055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ee690-7c66-44b2-956d-2183711b4675 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.061653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ee690-7c66-44b2-956d-2183711b4675 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.079576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:59.117113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b011f-22f9-43dc-b1da-2fc76bc85c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.117767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b011f-22f9-43dc-b1da-2fc76bc85c2f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.118223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b011f-22f9-43dc-b1da-2fc76bc85c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.119585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b011f-22f9-43dc-b1da-2fc76bc85c2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.121751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b011f-22f9-43dc-b1da-2fc76bc85c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:59.124164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b011f-22f9-43dc-b1da-2fc76bc85c2f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:59.126200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 64/142] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:59.131890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cecb7f-1d9e-4ebf-b5ed-a52f3cdf4958 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.203391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b011f-22f9-43dc-b1da-2fc76bc85c2f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.334054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cecb7f-1d9e-4ebf-b5ed-a52f3cdf4958 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.398632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e10e4-0896-4754-9120-72eaef47c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.399879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e10e4-0896-4754-9120-72eaef47c4ce None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.400410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e10e4-0896-4754-9120-72eaef47c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.400971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e10e4-0896-4754-9120-72eaef47c4ce None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.401838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e10e4-0896-4754-9120-72eaef47c4ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:59.403611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e10e4-0896-4754-9120-72eaef47c4ce None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:59.440215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b011f-22f9-43dc-b1da-2fc76bc85c2f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.485099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6691a-e4c5-4a44-9248-e3cda0947be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.485829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6691a-e4c5-4a44-9248-e3cda0947be5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.486149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6691a-e4c5-4a44-9248-e3cda0947be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.486430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6691a-e4c5-4a44-9248-e3cda0947be5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.500644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6691a-e4c5-4a44-9248-e3cda0947be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-88d6691a-e4c5-4a44-9248-e3cda0947be5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:59.506933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:59.507824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:28:59.508613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 65/143] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:59.514522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6691a-e4c5-4a44-9248-e3cda0947be5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.517601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 79/144] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:28:59 2022] POST /identity/v3/users => generated 283 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:59.572640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:59.642674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6691a-e4c5-4a44-9248-e3cda0947be5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.683021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.683671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.683960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.684242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.684954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:59.685265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:59.686745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:59.697950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 66/145] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:28:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:59.704147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e10e4-0896-4754-9120-72eaef47c4ce None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.727649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:59.837140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e10e4-0896-4754-9120-72eaef47c4ce None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.881323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cd342-87a1-4f4e-a56b-17f9f41a5e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.881998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cd342-87a1-4f4e-a56b-17f9f41a5e0a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.882279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cd342-87a1-4f4e-a56b-17f9f41a5e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.882549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cd342-87a1-4f4e-a56b-17f9f41a5e0a None admin] PATH_INFO: `/v3/users/3eb0c19623d54fd7bc324b91920cf2dc` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.883822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cd342-87a1-4f4e-a56b-17f9f41a5e0a None admin] RBAC: Authorizing `identity:get_user(user_id=3eb0c19623d54fd7bc324b91920cf2dc)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:59.886294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cd342-87a1-4f4e-a56b-17f9f41a5e0a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:59.889036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 80/146] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:59 2022] GET /identity/v3/users/3eb0c19623d54fd7bc324b91920cf2dc => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:59.893650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:00.039053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a432-0a1b-4539-8cc9-53fa6a6c7743 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:00.041187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:00.085123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01557d8c-1858-4631-917b-cff8f0a348ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:00.085785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01557d8c-1858-4631-917b-cff8f0a348ce None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:00.086105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01557d8c-1858-4631-917b-cff8f0a348ce None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:00.086386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01557d8c-1858-4631-917b-cff8f0a348ce None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:00.088265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01557d8c-1858-4631-917b-cff8f0a348ce None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:00.089748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01557d8c-1858-4631-917b-cff8f0a348ce None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:00.091785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 67/147] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:59 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:00.097785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cd342-87a1-4f4e-a56b-17f9f41a5e0a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:00.218642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:00.238613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cd342-87a1-4f4e-a56b-17f9f41a5e0a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:00.279393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73b2ae-711e-4d85-a7bc-cc9b35c6e4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:00.280072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b2ae-711e-4d85-a7bc-cc9b35c6e4f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:00.280353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b2ae-711e-4d85-a7bc-cc9b35c6e4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:00.280636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b2ae-711e-4d85-a7bc-cc9b35c6e4f8 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202/users/3eb0c19623d54fd7bc324b91920cf2dc/roles/f9e4fcd3595c43609a8c949b1996b02c` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:00.284670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73b2ae-711e-4d85-a7bc-cc9b35c6e4f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f9e413c56d445ab804ef9a518cc2202, user_id=3eb0c19623d54fd7bc324b91920cf2dc, role_id=f9e4fcd3595c43609a8c949b1996b02c)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:00.286206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73b2ae-711e-4d85-a7bc-cc9b35c6e4f8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:00.372757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 81/148] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:00 2022] PUT /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202/users/3eb0c19623d54fd7bc324b91920cf2dc/roles/f9e4fcd3595c43609a8c949b1996b02c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:00.967416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c699b1d-bc27-47c0-8944-40efc5c58881 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:00.972071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c699b1d-bc27-47c0-8944-40efc5c58881 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:00.972071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c699b1d-bc27-47c0-8944-40efc5c58881 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:00.972071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 68/149] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:00.976120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1471e-b98b-497a-8eeb-097876ab8d01 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:00.976638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1471e-b98b-497a-8eeb-097876ab8d01 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:00.977099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1471e-b98b-497a-8eeb-097876ab8d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.007478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-15b1471e-b98b-497a-8eeb-097876ab8d01 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:01.009779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-15b1471e-b98b-497a-8eeb-097876ab8d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:01.105124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 82/150] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:01.111081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af3d4-f3dd-4ecb-912a-fafb57ddc9cb None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.111375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af3d4-f3dd-4ecb-912a-fafb57ddc9cb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.111665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af3d4-f3dd-4ecb-912a-fafb57ddc9cb None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.113185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 69/151] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:01.119660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21de145-6b19-4d99-a0a2-2ef5b14761c7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.120132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21de145-6b19-4d99-a0a2-2ef5b14761c7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.120413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21de145-6b19-4d99-a0a2-2ef5b14761c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.147160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a21de145-6b19-4d99-a0a2-2ef5b14761c7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:01.148405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a21de145-6b19-4d99-a0a2-2ef5b14761c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:01.235763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 83/152] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:01.357409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776bccc-08ea-478c-849b-fd95a6eb6cac None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.357772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776bccc-08ea-478c-849b-fd95a6eb6cac None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.358058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776bccc-08ea-478c-849b-fd95a6eb6cac None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.359660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 70/153] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:01.367632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6000c41-b784-457c-92d3-660abc962da1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.367929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6000c41-b784-457c-92d3-660abc962da1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.368194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6000c41-b784-457c-92d3-660abc962da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.383428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:01.394419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d6000c41-b784-457c-92d3-660abc962da1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:01.397498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d6000c41-b784-457c-92d3-660abc962da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:01.473824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776bccc-08ea-478c-849b-fd95a6eb6cac None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:01.517343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 84/154] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:01.658406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776bccc-08ea-478c-849b-fd95a6eb6cac None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:01.705776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5ab3b-6de5-41d1-bbad-65fc61a8394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:01.706516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5ab3b-6de5-41d1-bbad-65fc61a8394f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.706874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5ab3b-6de5-41d1-bbad-65fc61a8394f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.707308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5ab3b-6de5-41d1-bbad-65fc61a8394f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.726691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5ab3b-6de5-41d1-bbad-65fc61a8394f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c8e5ab3b-6de5-41d1-bbad-65fc61a8394f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:01.732068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:01.732930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:01.734160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 71/155] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:01.739394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6000c41-b784-457c-92d3-660abc962da1 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:01.742979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5ab3b-6de5-41d1-bbad-65fc61a8394f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:01.787946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:01.866827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6000c41-b784-457c-92d3-660abc962da1 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:01.909427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b3500-a614-4489-80d0-5583e03e1f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:01.910337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b3500-a614-4489-80d0-5583e03e1f49 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.910717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b3500-a614-4489-80d0-5583e03e1f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.911632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b3500-a614-4489-80d0-5583e03e1f49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.924483 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:01.925667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b3500-a614-4489-80d0-5583e03e1f49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-0d0b3500-a614-4489-80d0-5583e03e1f49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:01.932854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:01.933908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:01.934722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 85/156] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:01.944756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5ab3b-6de5-41d1-bbad-65fc61a8394f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:01.967459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b3500-a614-4489-80d0-5583e03e1f49 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.027696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.028951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.029492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.030006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.030931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:02.031447 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.039539 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.053476 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 72/157] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:02.060803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.108221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b3500-a614-4489-80d0-5583e03e1f49 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.151859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.152480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.152756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.153026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.153702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:02.154040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.156047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.168027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 86/158] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:02.178903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.247322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.255948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.266634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d23ef-3d37-4c8d-a3e9-d585c986b510 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.309254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3ed3-b7f4-492d-b426-a329eda8169d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.339762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ff6ae-e7b6-45d5-931d-404d48a2203f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.340654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ff6ae-e7b6-45d5-931d-404d48a2203f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.341076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ff6ae-e7b6-45d5-931d-404d48a2203f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.341485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ff6ae-e7b6-45d5-931d-404d48a2203f None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.346714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ff6ae-e7b6-45d5-931d-404d48a2203f None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.348385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ff6ae-e7b6-45d5-931d-404d48a2203f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.350546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 73/159] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:02.351537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ff6ae-e7b6-45d5-931d-404d48a2203f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.355369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc506c2-bf00-4423-8547-6cc195584311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.356526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc506c2-bf00-4423-8547-6cc195584311 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.356959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc506c2-bf00-4423-8547-6cc195584311 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.357334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc506c2-bf00-4423-8547-6cc195584311 None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.359869 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc506c2-bf00-4423-8547-6cc195584311 None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.361517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc506c2-bf00-4423-8547-6cc195584311 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.364741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 87/160] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:02.375028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc506c2-bf00-4423-8547-6cc195584311 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.554627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc506c2-bf00-4423-8547-6cc195584311 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.563620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ff6ae-e7b6-45d5-931d-404d48a2203f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.619761 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.626843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482c819-a894-4900-948e-e0cf06897c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.629521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482c819-a894-4900-948e-e0cf06897c9a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.629521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482c819-a894-4900-948e-e0cf06897c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.629521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482c819-a894-4900-948e-e0cf06897c9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.639953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470d214-dfd2-4e31-93df-1ed670641b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.640829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470d214-dfd2-4e31-93df-1ed670641b38 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.641331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470d214-dfd2-4e31-93df-1ed670641b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.641742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470d214-dfd2-4e31-93df-1ed670641b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.657761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482c819-a894-4900-948e-e0cf06897c9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-b482c819-a894-4900-948e-e0cf06897c9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:02.664023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:02.665125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:02.666910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:02.670704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 74/161] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:02.673210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482c819-a894-4900-948e-e0cf06897c9a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.719512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.782061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470d214-dfd2-4e31-93df-1ed670641b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.783546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470d214-dfd2-4e31-93df-1ed670641b38 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.884068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482c819-a894-4900-948e-e0cf06897c9a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.946131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7906-47ea-4d15-b2c0-4058334f9e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.947504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7906-47ea-4d15-b2c0-4058334f9e6a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.948048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7906-47ea-4d15-b2c0-4058334f9e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.948515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7906-47ea-4d15-b2c0-4058334f9e6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.037501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 88/162] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:03.038839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271238-cd91-4645-b2d5-c4cfcdd5ff2d None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.039128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271238-cd91-4645-b2d5-c4cfcdd5ff2d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.039406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271238-cd91-4645-b2d5-c4cfcdd5ff2d None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.049763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 89/163] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:03.050739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26271238-cd91-4645-b2d5-c4cfcdd5ff2d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.052429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb7906-47ea-4d15-b2c0-4058334f9e6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.054926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb7906-47ea-4d15-b2c0-4058334f9e6a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.093943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.131676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.270794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 75/164] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:03.273115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7906-47ea-4d15-b2c0-4058334f9e6a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.331650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26271238-cd91-4645-b2d5-c4cfcdd5ff2d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.396013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b408507-8b35-41e5-8429-8c4b12ba9a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.396671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408507-8b35-41e5-8429-8c4b12ba9a77 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.396956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408507-8b35-41e5-8429-8c4b12ba9a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.397234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408507-8b35-41e5-8429-8c4b12ba9a77 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.397964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b408507-8b35-41e5-8429-8c4b12ba9a77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:03.398265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b408507-8b35-41e5-8429-8c4b12ba9a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.399765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b408507-8b35-41e5-8429-8c4b12ba9a77 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.413289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 90/165] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:03.417343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c11df8-3d88-4218-a93a-a1149a51b5b3 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.418331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c11df8-3d88-4218-a93a-a1149a51b5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.418624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c11df8-3d88-4218-a93a-a1149a51b5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.458991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-c1c11df8-3d88-4218-a93a-a1149a51b5b3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:03.460214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c1c11df8-3d88-4218-a93a-a1149a51b5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:03.492703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7906-47ea-4d15-b2c0-4058334f9e6a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.538849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.574625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1836472-fb8e-4129-bd71-2f6777945ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.575284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1836472-fb8e-4129-bd71-2f6777945ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.575570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1836472-fb8e-4129-bd71-2f6777945ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.575909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1836472-fb8e-4129-bd71-2f6777945ee4 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.577180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1836472-fb8e-4129-bd71-2f6777945ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.578702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1836472-fb8e-4129-bd71-2f6777945ee4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.580670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 76/166] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:03.582497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1836472-fb8e-4129-bd71-2f6777945ee4 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.584205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 91/167] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:03.585072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c11df8-3d88-4218-a93a-a1149a51b5b3 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.616463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.747955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c11df8-3d88-4218-a93a-a1149a51b5b3 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.800817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1836472-fb8e-4129-bd71-2f6777945ee4 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.811146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2e12a-3b6a-46f0-8351-63d7f61a54c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.811797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e12a-3b6a-46f0-8351-63d7f61a54c4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.812077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e12a-3b6a-46f0-8351-63d7f61a54c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.812347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e12a-3b6a-46f0-8351-63d7f61a54c4 None admin] PATH_INFO: `/v3/users/1d96bcf28939461487e0fd8a9e516811` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.818588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2e12a-3b6a-46f0-8351-63d7f61a54c4 None admin] RBAC: Authorizing `identity:get_user(user_id=1d96bcf28939461487e0fd8a9e516811)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.820071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2e12a-3b6a-46f0-8351-63d7f61a54c4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.822568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 92/168] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/users/1d96bcf28939461487e0fd8a9e516811 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:03.823167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2e12a-3b6a-46f0-8351-63d7f61a54c4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.851466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f799fad-8d9d-4440-881b-525cf1e98fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.852707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f799fad-8d9d-4440-881b-525cf1e98fcf None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.853203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f799fad-8d9d-4440-881b-525cf1e98fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.853776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f799fad-8d9d-4440-881b-525cf1e98fcf None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.855217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f799fad-8d9d-4440-881b-525cf1e98fcf None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.857010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f799fad-8d9d-4440-881b-525cf1e98fcf None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.858946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 77/169] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:03.860048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f799fad-8d9d-4440-881b-525cf1e98fcf None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.935817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.945407 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.974249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2e12a-3b6a-46f0-8351-63d7f61a54c4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.988311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f799fad-8d9d-4440-881b-525cf1e98fcf None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:04.031259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376765fa-de5e-4a7d-804c-ca2dea48995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:04.033370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376765fa-de5e-4a7d-804c-ca2dea48995f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.033906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376765fa-de5e-4a7d-804c-ca2dea48995f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.034344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376765fa-de5e-4a7d-804c-ca2dea48995f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.046627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:04.047215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.047493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.047764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.048506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] RBAC: Adding query filter params (role.id=fff136f214f2428895daf10483c542d1, scope.project.id=2f9e413c56d445ab804ef9a518cc2202, user.id=1d96bcf28939461487e0fd8a9e516811) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:04.048805 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:04.050307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:04.057973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 93/170] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/role_assignments?user.id=1d96bcf28939461487e0fd8a9e516811&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=fff136f214f2428895daf10483c542d1 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:04.058789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:04.145582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376765fa-de5e-4a7d-804c-ca2dea48995f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:04.147931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376765fa-de5e-4a7d-804c-ca2dea48995f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:04.240822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:04.268783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d86dd-a868-4543-9dd6-febe5dd28c35 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:04.335815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 78/171] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:04.337209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55094d0b-ad6f-4092-bc86-c86228c9c70f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.337499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55094d0b-ad6f-4092-bc86-c86228c9c70f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.337785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55094d0b-ad6f-4092-bc86-c86228c9c70f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.339243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 79/172] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:04.340521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a544d6b-af70-437a-bb70-5bd0f90e9294 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.340834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a544d6b-af70-437a-bb70-5bd0f90e9294 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.341160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a544d6b-af70-437a-bb70-5bd0f90e9294 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.345810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:04.346453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.346828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.347138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.347900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] RBAC: Adding query filter params (role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=2f9e413c56d445ab804ef9a518cc2202, user.id=1d96bcf28939461487e0fd8a9e516811) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:04.348237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:04.349725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:04.353040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 80/173] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:04.353772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a544d6b-af70-437a-bb70-5bd0f90e9294 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:04.366318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 94/174] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:04 2022] GET /identity/v3/role_assignments?user.id=1d96bcf28939461487e0fd8a9e516811&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=d6fe1aca91e74aad9907c855501f588f => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:04.367568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:04.402244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:04.558874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a544d6b-af70-437a-bb70-5bd0f90e9294 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:04.603299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21baf137-a8f6-4328-9db7-89e4cef8ebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:04.604297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baf137-a8f6-4328-9db7-89e4cef8ebfd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.604797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baf137-a8f6-4328-9db7-89e4cef8ebfd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.605319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baf137-a8f6-4328-9db7-89e4cef8ebfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.605921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aed244-b4fd-47a5-87ad-485fe9e13db6 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:04.623633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21baf137-a8f6-4328-9db7-89e4cef8ebfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-21baf137-a8f6-4328-9db7-89e4cef8ebfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:04.629149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:04.630138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:04.630915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:04.633347 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 81/175] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:04.635662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7728d-b480-475a-bbdc-1c7d6a9e76de None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.636170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7728d-b480-475a-bbdc-1c7d6a9e76de None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.636657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7728d-b480-475a-bbdc-1c7d6a9e76de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.648489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6a95a-cb0d-4d2b-bca6-986a52a49175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:04.649139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6a95a-cb0d-4d2b-bca6-986a52a49175 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.649497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6a95a-cb0d-4d2b-bca6-986a52a49175 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.650336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6a95a-cb0d-4d2b-bca6-986a52a49175 None admin] PATH_INFO: `/v3/projects/af6939c5e5844d23a68c27694bdd64c0` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.652952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6a95a-cb0d-4d2b-bca6-986a52a49175 None admin] RBAC: Authorizing `identity:get_project(project_id=af6939c5e5844d23a68c27694bdd64c0)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:04.654465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6a95a-cb0d-4d2b-bca6-986a52a49175 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:04.656549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 95/176] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:04 2022] GET /identity/v3/projects/af6939c5e5844d23a68c27694bdd64c0 => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:04.657552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa70c5-5d5d-4946-8cdb-a2c46bde8957 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.657836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa70c5-5d5d-4946-8cdb-a2c46bde8957 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.657959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-0cf7728d-b480-475a-bbdc-1c7d6a9e76de None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:04.658278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa70c5-5d5d-4946-8cdb-a2c46bde8957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.659122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0cf7728d-b480-475a-bbdc-1c7d6a9e76de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:04.674279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:04.686567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-70aa70c5-5d5d-4946-8cdb-a2c46bde8957 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:04.687818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-70aa70c5-5d5d-4946-8cdb-a2c46bde8957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:04.755960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 82/177] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:04.757683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb105e-bd25-469d-a573-d0e002cca21c None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.758041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb105e-bd25-469d-a573-d0e002cca21c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.758325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb105e-bd25-469d-a573-d0e002cca21c None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.762019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 83/178] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:04.762864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb105e-bd25-469d-a573-d0e002cca21c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:04.780187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 96/179] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:04.780828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa70c5-5d5d-4946-8cdb-a2c46bde8957 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:04.796392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:04.892139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa70c5-5d5d-4946-8cdb-a2c46bde8957 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:04.934139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3bc96-aa59-4498-85cf-9d4bcd612127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:04.934825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3bc96-aa59-4498-85cf-9d4bcd612127 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.935110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3bc96-aa59-4498-85cf-9d4bcd612127 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.935437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3bc96-aa59-4498-85cf-9d4bcd612127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.936192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3bc96-aa59-4498-85cf-9d4bcd612127 None admin] RBAC: Adding query filter params (role.id=fff136f214f2428895daf10483c542d1, scope.project.id=af6939c5e5844d23a68c27694bdd64c0, user.id=1d96bcf28939461487e0fd8a9e516811) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:04.936506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3bc96-aa59-4498-85cf-9d4bcd612127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:04.938681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3bc96-aa59-4498-85cf-9d4bcd612127 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:04.941081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb105e-bd25-469d-a573-d0e002cca21c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:04.945486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 97/180] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:04 2022] GET /identity/v3/role_assignments?user.id=1d96bcf28939461487e0fd8a9e516811&scope.project.id=af6939c5e5844d23a68c27694bdd64c0&role.id=fff136f214f2428895daf10483c542d1 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:04.947241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f532a6-8822-4c3b-97b3-03bb1583f56d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:04.947528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f532a6-8822-4c3b-97b3-03bb1583f56d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:04.947923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f532a6-8822-4c3b-97b3-03bb1583f56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:04.977233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-08f532a6-8822-4c3b-97b3-03bb1583f56d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:04.979058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-08f532a6-8822-4c3b-97b3-03bb1583f56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:05.026732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e185dd-fb20-4b9f-bec2-42f7275e47e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.027385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e185dd-fb20-4b9f-bec2-42f7275e47e9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.027649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e185dd-fb20-4b9f-bec2-42f7275e47e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.028053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e185dd-fb20-4b9f-bec2-42f7275e47e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.028613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e185dd-fb20-4b9f-bec2-42f7275e47e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:05.029158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e185dd-fb20-4b9f-bec2-42f7275e47e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:05.030377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e185dd-fb20-4b9f-bec2-42f7275e47e9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:05.055496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 84/181] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:05.057496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58d3e-84f8-4214-bb53-7c05691a5431 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.057783 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58d3e-84f8-4214-bb53-7c05691a5431 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.058089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58d3e-84f8-4214-bb53-7c05691a5431 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.059966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 85/182] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:05.071899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a41e-4121-4aff-943f-6f7bcf96a052 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.072228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a41e-4121-4aff-943f-6f7bcf96a052 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.072549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a41e-4121-4aff-943f-6f7bcf96a052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.096149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 98/183] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:05.097504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f532a6-8822-4c3b-97b3-03bb1583f56d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.114627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.116291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4891a41e-4121-4aff-943f-6f7bcf96a052 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:05.117215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4891a41e-4121-4aff-943f-6f7bcf96a052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:05.122297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.232158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 86/184] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:05.233143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891a41e-4121-4aff-943f-6f7bcf96a052 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.274017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f532a6-8822-4c3b-97b3-03bb1583f56d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:05.318211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1d437-d55d-4178-bf22-4b89054edf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.319368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1d437-d55d-4178-bf22-4b89054edf47 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.319888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1d437-d55d-4178-bf22-4b89054edf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.320516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1d437-d55d-4178-bf22-4b89054edf47 None admin] PATH_INFO: `/v3/users/9c52ecdd884449e5b0fb195998b79885` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.322636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1d437-d55d-4178-bf22-4b89054edf47 None admin] RBAC: Authorizing `identity:get_user(user_id=9c52ecdd884449e5b0fb195998b79885)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:05.326191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1d437-d55d-4178-bf22-4b89054edf47 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:05.328734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 99/185] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/users/9c52ecdd884449e5b0fb195998b79885 => generated 247 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:05.330133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1d437-d55d-4178-bf22-4b89054edf47 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.358126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891a41e-4121-4aff-943f-6f7bcf96a052 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:05.397426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4791c7-0901-4443-b4b4-25ab40b8fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.398116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4791c7-0901-4443-b4b4-25ab40b8fd7f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.398399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4791c7-0901-4443-b4b4-25ab40b8fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.398686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4791c7-0901-4443-b4b4-25ab40b8fd7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.407028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.410964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4791c7-0901-4443-b4b4-25ab40b8fd7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-6a4791c7-0901-4443-b4b4-25ab40b8fd7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:05.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:05.417454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:05.418288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 87/186] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:05.418969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4791c7-0901-4443-b4b4-25ab40b8fd7f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.441032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.476533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1d437-d55d-4178-bf22-4b89054edf47 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:05.524701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd1508-a17a-4914-818b-ffea4f613fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.525599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd1508-a17a-4914-818b-ffea4f613fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.525915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd1508-a17a-4914-818b-ffea4f613fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.526234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd1508-a17a-4914-818b-ffea4f613fc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.537737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd1508-a17a-4914-818b-ffea4f613fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:05.538694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4791c7-0901-4443-b4b4-25ab40b8fd7f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-06fd1508-a17a-4914-818b-ffea4f613fc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:05.544459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:05.545482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:05.546772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:05.548154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 100/187] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:05.548528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd1508-a17a-4914-818b-ffea4f613fc5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.582196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a65e07-2772-4ba0-a90d-d81277070e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.582815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65e07-2772-4ba0-a90d-d81277070e7d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.583093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65e07-2772-4ba0-a90d-d81277070e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.583365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65e07-2772-4ba0-a90d-d81277070e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.652894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.657405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a65e07-2772-4ba0-a90d-d81277070e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:05.658998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a65e07-2772-4ba0-a90d-d81277070e7d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:05.666465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd1508-a17a-4914-818b-ffea4f613fc5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:05.702816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c857b5-4c86-4dd4-baab-ffaf266859a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.703448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c857b5-4c86-4dd4-baab-ffaf266859a5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.703746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c857b5-4c86-4dd4-baab-ffaf266859a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.704030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c857b5-4c86-4dd4-baab-ffaf266859a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.706127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.716048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c857b5-4c86-4dd4-baab-ffaf266859a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-75c857b5-4c86-4dd4-baab-ffaf266859a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:05.720816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:05.721904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:05.722738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 101/188] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:05.723396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c857b5-4c86-4dd4-baab-ffaf266859a5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.772725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 88/189] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:05.773678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a65e07-2772-4ba0-a90d-d81277070e7d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.838631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c857b5-4c86-4dd4-baab-ffaf266859a5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:05.874642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.875255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.875542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.875817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.876269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.876501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:05.876806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:05.878247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:05.880606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a65e07-2772-4ba0-a90d-d81277070e7d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:05.889981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 102/190] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:05.894623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.924530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.925143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.925424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.925689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.926397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:05.926702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:05.928120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:05.937029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 89/191] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:05.937882 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.960324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:06.006253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b487f-3df5-4f55-97cd-618d69694c29 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.046451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f12e1-07bd-40aa-a9e3-c933ea00f32e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.090322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.090857 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.091142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.091415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.092097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:06.092401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.093793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.100949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd1804-4196-4f5d-88bb-cddf0e297299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.101841 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1804-4196-4f5d-88bb-cddf0e297299 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.102148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1804-4196-4f5d-88bb-cddf0e297299 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.102456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1804-4196-4f5d-88bb-cddf0e297299 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.103559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 90/192] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:06.104355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:06.106432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd1804-4196-4f5d-88bb-cddf0e297299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.109413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd1804-4196-4f5d-88bb-cddf0e297299 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.111277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 103/193] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:06.114422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd1804-4196-4f5d-88bb-cddf0e297299 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:06.131169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:06.210744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec9312-1eca-4755-b4bb-0bb134085c7a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.225785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd1804-4196-4f5d-88bb-cddf0e297299 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.247504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaec92-0cf0-40a2-a90d-0a924aba6ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.248136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaec92-0cf0-40a2-a90d-0a924aba6ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.248469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaec92-0cf0-40a2-a90d-0a924aba6ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.248769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaec92-0cf0-40a2-a90d-0a924aba6ee3 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.250341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaec92-0cf0-40a2-a90d-0a924aba6ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.251490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:06.251818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaec92-0cf0-40a2-a90d-0a924aba6ee3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.253916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 91/194] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:06 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:06.254733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaec92-0cf0-40a2-a90d-0a924aba6ee3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:06.268255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2651a-cbb7-4582-bbb6-be6f582e21a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.268923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2651a-cbb7-4582-bbb6-be6f582e21a5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.269209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2651a-cbb7-4582-bbb6-be6f582e21a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.269543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2651a-cbb7-4582-bbb6-be6f582e21a5 None admin] PATH_INFO: `/v3/users/6e154371f45e4cb081bd3c705ba1b9fb` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.283930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2651a-cbb7-4582-bbb6-be6f582e21a5 None admin] RBAC: Authorizing `identity:get_user(user_id=6e154371f45e4cb081bd3c705ba1b9fb)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.286168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2651a-cbb7-4582-bbb6-be6f582e21a5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.288125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 104/195] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:06 2022] GET /identity/v3/users/6e154371f45e4cb081bd3c705ba1b9fb => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:06.289810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2651a-cbb7-4582-bbb6-be6f582e21a5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:06.355499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:06.364877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaec92-0cf0-40a2-a90d-0a924aba6ee3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.400719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2651a-cbb7-4582-bbb6-be6f582e21a5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.401670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d3acf-5058-47af-be37-f4c9018c727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.402342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d3acf-5058-47af-be37-f4c9018c727e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.402620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d3acf-5058-47af-be37-f4c9018c727e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.402895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d3acf-5058-47af-be37-f4c9018c727e None admin] PATH_INFO: `/v3/users/3eb0c19623d54fd7bc324b91920cf2dc` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.404039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d3acf-5058-47af-be37-f4c9018c727e None admin] RBAC: Authorizing `identity:get_user(user_id=3eb0c19623d54fd7bc324b91920cf2dc)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.405463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d3acf-5058-47af-be37-f4c9018c727e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.407097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 92/196] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:06 2022] GET /identity/v3/users/3eb0c19623d54fd7bc324b91920cf2dc => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:06.407933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d3acf-5058-47af-be37-f4c9018c727e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:06.443178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d477c-c90c-4d6b-9e52-978518c3ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.443811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d477c-c90c-4d6b-9e52-978518c3ae43 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.444999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d477c-c90c-4d6b-9e52-978518c3ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.444999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d477c-c90c-4d6b-9e52-978518c3ae43 None admin] PATH_INFO: `/v3/users/6e154371f45e4cb081bd3c705ba1b9fb` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.449836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d477c-c90c-4d6b-9e52-978518c3ae43 None admin] RBAC: Authorizing `identity:get_user(user_id=6e154371f45e4cb081bd3c705ba1b9fb)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.451380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d477c-c90c-4d6b-9e52-978518c3ae43 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.453015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 105/197] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:06 2022] GET /identity/v3/users/6e154371f45e4cb081bd3c705ba1b9fb => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:06.454038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d477c-c90c-4d6b-9e52-978518c3ae43 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:06.508445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:06.518618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d3acf-5058-47af-be37-f4c9018c727e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.553003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31440-0a67-4415-b19a-d129ba9c479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.553578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31440-0a67-4415-b19a-d129ba9c479b None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.553877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31440-0a67-4415-b19a-d129ba9c479b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.554159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31440-0a67-4415-b19a-d129ba9c479b None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.554812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b31440-0a67-4415-b19a-d129ba9c479b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.556810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b31440-0a67-4415-b19a-d129ba9c479b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.564256 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d477c-c90c-4d6b-9e52-978518c3ae43 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.601996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.603150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.603640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.604026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.604763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=2f9e413c56d445ab804ef9a518cc2202, user.id=9c52ecdd884449e5b0fb195998b79885) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:06.605065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.606640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.612521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 106/198] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:06 2022] GET /identity/v3/role_assignments?user.id=9c52ecdd884449e5b0fb195998b79885&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:06.613377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:06.693585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 93/199] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:06 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:06.694322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31440-0a67-4415-b19a-d129ba9c479b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:06.715385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:06.765836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0371-a05a-4315-91fe-5e05be1fd0f2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.794287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:06.822253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64144e-ad95-456a-ba2a-3f599e5fbec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.823346 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64144e-ad95-456a-ba2a-3f599e5fbec0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.823638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64144e-ad95-456a-ba2a-3f599e5fbec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.823920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64144e-ad95-456a-ba2a-3f599e5fbec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.840698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31440-0a67-4415-b19a-d129ba9c479b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.900120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6ff25-0745-4df2-91ab-63b85cc914c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.901572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6ff25-0745-4df2-91ab-63b85cc914c7 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.902041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6ff25-0745-4df2-91ab-63b85cc914c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.902485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6ff25-0745-4df2-91ab-63b85cc914c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.934300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64144e-ad95-456a-ba2a-3f599e5fbec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.936347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64144e-ad95-456a-ba2a-3f599e5fbec0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:07.006069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6ff25-0745-4df2-91ab-63b85cc914c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:07.007693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6ff25-0745-4df2-91ab-63b85cc914c7 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:07.031383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:07.074962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 107/200] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:07.075977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64144e-ad95-456a-ba2a-3f599e5fbec0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:07.095675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:07.158120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 94/201] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:07.159018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6ff25-0745-4df2-91ab-63b85cc914c7 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:07.210770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64144e-ad95-456a-ba2a-3f599e5fbec0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:07.263589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217471f9-4e44-4024-b462-fa6209cffa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:07.264222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217471f9-4e44-4024-b462-fa6209cffa27 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:07.264505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217471f9-4e44-4024-b462-fa6209cffa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:07.264805 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217471f9-4e44-4024-b462-fa6209cffa27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:07.283230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6ff25-0745-4df2-91ab-63b85cc914c7 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:07.301458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:07.333284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990a9e3-2cdf-439f-8066-693169e7b082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:07.334032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990a9e3-2cdf-439f-8066-693169e7b082 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:07.334351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990a9e3-2cdf-439f-8066-693169e7b082 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:07.334660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990a9e3-2cdf-439f-8066-693169e7b082 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:07.336477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990a9e3-2cdf-439f-8066-693169e7b082 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:07.338456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990a9e3-2cdf-439f-8066-693169e7b082 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:07.345345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 95/202] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:07 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:07.345345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990a9e3-2cdf-439f-8066-693169e7b082 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:07.365885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217471f9-4e44-4024-b462-fa6209cffa27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:07.365885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217471f9-4e44-4024-b462-fa6209cffa27 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:07.386265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:07.471935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990a9e3-2cdf-439f-8066-693169e7b082 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:07.475736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 108/203] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:07.481396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217471f9-4e44-4024-b462-fa6209cffa27 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:07.510647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a86b-1754-4bea-a887-3e7494e2b186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:07.511402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a86b-1754-4bea-a887-3e7494e2b186 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:07.511688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a86b-1754-4bea-a887-3e7494e2b186 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:07.511960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a86b-1754-4bea-a887-3e7494e2b186 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:07.513107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8a86b-1754-4bea-a887-3e7494e2b186 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:07.514829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8a86b-1754-4bea-a887-3e7494e2b186 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:07.516676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 96/204] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:07 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:07.522488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a86b-1754-4bea-a887-3e7494e2b186 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:07.578410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:07.599291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217471f9-4e44-4024-b462-fa6209cffa27 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:07.615807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:07.642062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a86b-1754-4bea-a887-3e7494e2b186 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:07.647806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:07.648708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:07.649869 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:07.650779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:07.651957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=a667e49b60584e6382b48fec8d858ac4, user.id=6e154371f45e4cb081bd3c705ba1b9fb) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:07.652527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:07.654218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:07.662361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 109/205] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:07 2022] GET /identity/v3/role_assignments?user.id=6e154371f45e4cb081bd3c705ba1b9fb&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:07.663596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:07.727883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b7fc-5b65-4bdf-99a7-5ecfb5355fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:07.728448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b7fc-5b65-4bdf-99a7-5ecfb5355fad None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:07.728730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b7fc-5b65-4bdf-99a7-5ecfb5355fad None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:07.729005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b7fc-5b65-4bdf-99a7-5ecfb5355fad None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:07.734681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901b7fc-5b65-4bdf-99a7-5ecfb5355fad None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:07.736177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901b7fc-5b65-4bdf-99a7-5ecfb5355fad None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:07.737936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 97/206] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:07 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:07.745150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b7fc-5b65-4bdf-99a7-5ecfb5355fad None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:07.813421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b146e-968c-4502-83f9-5653a49dad18 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:07.859704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d753a07-16d4-40ad-990a-cdcc205ac15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:07.860326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d753a07-16d4-40ad-990a-cdcc205ac15c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:07.861221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d753a07-16d4-40ad-990a-cdcc205ac15c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:07.861541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d753a07-16d4-40ad-990a-cdcc205ac15c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:07.862378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d753a07-16d4-40ad-990a-cdcc205ac15c None admin] RBAC: Adding query filter params (role.id=f9e4fcd3595c43609a8c949b1996b02c, scope.project.id=2f9e413c56d445ab804ef9a518cc2202, user.id=3eb0c19623d54fd7bc324b91920cf2dc) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:07.862700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d753a07-16d4-40ad-990a-cdcc205ac15c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:07.864274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d753a07-16d4-40ad-990a-cdcc205ac15c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:07.872230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 110/207] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:07 2022] GET /identity/v3/role_assignments?user.id=3eb0c19623d54fd7bc324b91920cf2dc&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=f9e4fcd3595c43609a8c949b1996b02c => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:07.970739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b7fc-5b65-4bdf-99a7-5ecfb5355fad None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:07.998650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:08.027845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d7ba6-fbfa-443d-986f-3e81b96b65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:08.028499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d7ba6-fbfa-443d-986f-3e81b96b65b4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:08.028781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d7ba6-fbfa-443d-986f-3e81b96b65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:08.029053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d7ba6-fbfa-443d-986f-3e81b96b65b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:08.029881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d7ba6-fbfa-443d-986f-3e81b96b65b4 None admin] RBAC: Adding query filter params (role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=a667e49b60584e6382b48fec8d858ac4, user.id=6e154371f45e4cb081bd3c705ba1b9fb) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:08.030204 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d7ba6-fbfa-443d-986f-3e81b96b65b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:08.031751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d7ba6-fbfa-443d-986f-3e81b96b65b4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:08.043032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 98/208] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:07 2022] GET /identity/v3/role_assignments?user.id=6e154371f45e4cb081bd3c705ba1b9fb&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=d6fe1aca91e74aad9907c855501f588f => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:08.973744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd50fd-4bdc-405e-a1f5-5f2054872e07 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:08.974415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd50fd-4bdc-405e-a1f5-5f2054872e07 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:08.974816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd50fd-4bdc-405e-a1f5-5f2054872e07 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:08.978080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 111/209] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:08.983920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f510a2f-7f14-496e-a94d-d37474232c09 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:08.984803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f510a2f-7f14-496e-a94d-d37474232c09 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:08.985325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f510a2f-7f14-496e-a94d-d37474232c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.050181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1f510a2f-7f14-496e-a94d-d37474232c09 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:09.051738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1f510a2f-7f14-496e-a94d-d37474232c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:09.068974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8513f8-d88a-4ec8-b657-f78c2a42c8e6 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.069350 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8513f8-d88a-4ec8-b657-f78c2a42c8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.069588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8513f8-d88a-4ec8-b657-f78c2a42c8e6 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.071948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 112/210] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:09.077298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad15f-4fd8-457a-b632-8b943a622fbb None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.077587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad15f-4fd8-457a-b632-8b943a622fbb None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.077879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad15f-4fd8-457a-b632-8b943a622fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.096676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b00ad15f-4fd8-457a-b632-8b943a622fbb None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:09.098146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b00ad15f-4fd8-457a-b632-8b943a622fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:09.136517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:09.145349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 99/211] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:09.176499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 113/212] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:09.437408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f510a2f-7f14-496e-a94d-d37474232c09 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:09.457012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ad15f-4fd8-457a-b632-8b943a622fbb None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:09.579908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f510a2f-7f14-496e-a94d-d37474232c09 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:09.602083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:09.622931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ad15f-4fd8-457a-b632-8b943a622fbb None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:09.631455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2bcd5-be2d-4f82-95fc-2842866ed439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:09.633120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2bcd5-be2d-4f82-95fc-2842866ed439 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.637054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2bcd5-be2d-4f82-95fc-2842866ed439 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.637330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2bcd5-be2d-4f82-95fc-2842866ed439 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.654831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2bcd5-be2d-4f82-95fc-2842866ed439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-deb2bcd5-be2d-4f82-95fc-2842866ed439 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:09.660917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:09.661826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:09.662655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:09.665231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 100/213] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:09.671759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8494-166c-4826-9c38-204b358c10d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:09.672061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2bcd5-be2d-4f82-95fc-2842866ed439 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:09.672411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8494-166c-4826-9c38-204b358c10d9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.672691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8494-166c-4826-9c38-204b358c10d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.672963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8494-166c-4826-9c38-204b358c10d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.674313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a8494-166c-4826-9c38-204b358c10d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:09.675765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a8494-166c-4826-9c38-204b358c10d9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:09.682004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 114/214] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:09.683573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8494-166c-4826-9c38-204b358c10d9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:09.730903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:09.792247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2bcd5-be2d-4f82-95fc-2842866ed439 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:09.796076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8494-166c-4826-9c38-204b358c10d9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:09.832830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5a702-4ee3-4901-bd95-534b7b6f2536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:09.833438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5a702-4ee3-4901-bd95-534b7b6f2536 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.833716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5a702-4ee3-4901-bd95-534b7b6f2536 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.834015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5a702-4ee3-4901-bd95-534b7b6f2536 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.834692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5a702-4ee3-4901-bd95-534b7b6f2536 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:09.834995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5a702-4ee3-4901-bd95-534b7b6f2536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:09.835102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5b77c-6722-46f4-8dc2-5fc138c6d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:09.835657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5b77c-6722-46f4-8dc2-5fc138c6d0a9 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.835943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5b77c-6722-46f4-8dc2-5fc138c6d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.836220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5b77c-6722-46f4-8dc2-5fc138c6d0a9 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.836397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5a702-4ee3-4901-bd95-534b7b6f2536 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:09.837542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5b77c-6722-46f4-8dc2-5fc138c6d0a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:09.839350 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5b77c-6722-46f4-8dc2-5fc138c6d0a9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:09.845912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 101/215] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:09.850198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644253bd-882a-4018-8d75-e844b770999f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.851375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644253bd-882a-4018-8d75-e844b770999f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.851774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644253bd-882a-4018-8d75-e844b770999f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.854057 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 102/216] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:09.854817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644253bd-882a-4018-8d75-e844b770999f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:09.892974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:09.954869 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 115/217] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:09 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:29:09.967163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7133-f312-479f-b065-986d80d2856c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.967575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644253bd-882a-4018-8d75-e844b770999f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:09.968089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7133-f312-479f-b065-986d80d2856c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.968391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7133-f312-479f-b065-986d80d2856c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.995737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ac5f7133-f312-479f-b065-986d80d2856c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:09.997499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ac5f7133-f312-479f-b065-986d80d2856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:10.018543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f990b-341f-440c-bdd9-517fd2e7671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:10.019515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f990b-341f-440c-bdd9-517fd2e7671a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.019913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f990b-341f-440c-bdd9-517fd2e7671a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.021005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f990b-341f-440c-bdd9-517fd2e7671a None admin] PATH_INFO: `/v3/users/9b9557fd154c48c7bb3ff6fbe036503b` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.043752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f990b-341f-440c-bdd9-517fd2e7671a None admin] RBAC: Authorizing `identity:get_user(user_id=9b9557fd154c48c7bb3ff6fbe036503b)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:10.046038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f990b-341f-440c-bdd9-517fd2e7671a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:10.048163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 103/218] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:09 2022] GET /identity/v3/users/9b9557fd154c48c7bb3ff6fbe036503b => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:10.055070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f990b-341f-440c-bdd9-517fd2e7671a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:10.095597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 116/219] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:10.096921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cbd32-6276-429f-87c7-12d6eb7df8b2 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.097212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cbd32-6276-429f-87c7-12d6eb7df8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.097488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cbd32-6276-429f-87c7-12d6eb7df8b2 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.099181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 117/220] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:10.105168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96792e-43ee-4122-a9bc-b78dd568ecb4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.105526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96792e-43ee-4122-a9bc-b78dd568ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.105895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96792e-43ee-4122-a9bc-b78dd568ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.131539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9d96792e-43ee-4122-a9bc-b78dd568ecb4 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:10.131539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9d96792e-43ee-4122-a9bc-b78dd568ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:10.158348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:10.185975 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:10.202584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f990b-341f-440c-bdd9-517fd2e7671a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:10.242681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 118/221] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:10.252004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f44b-a53f-42c1-8f21-c25617d2396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:10.252557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f44b-a53f-42c1-8f21-c25617d2396e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.252833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f44b-a53f-42c1-8f21-c25617d2396e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.253105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f44b-a53f-42c1-8f21-c25617d2396e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.329285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56f44b-a53f-42c1-8f21-c25617d2396e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:10.330808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56f44b-a53f-42c1-8f21-c25617d2396e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:10.369087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96792e-43ee-4122-a9bc-b78dd568ecb4 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:10.499102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:10.505424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96792e-43ee-4122-a9bc-b78dd568ecb4 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:10.521561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 104/222] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:10.531457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f44b-a53f-42c1-8f21-c25617d2396e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:10.576528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893187dc-e77d-4cd6-8e2d-98560b637add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:10.577563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893187dc-e77d-4cd6-8e2d-98560b637add None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.578127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893187dc-e77d-4cd6-8e2d-98560b637add None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.578636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893187dc-e77d-4cd6-8e2d-98560b637add None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.594197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893187dc-e77d-4cd6-8e2d-98560b637add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-893187dc-e77d-4cd6-8e2d-98560b637add None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:10.599751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:10.600705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 119/223] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:10.604804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893187dc-e77d-4cd6-8e2d-98560b637add None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:10.628877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:10.694967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f44b-a53f-42c1-8f21-c25617d2396e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:10.727166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893187dc-e77d-4cd6-8e2d-98560b637add None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:10.735096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a8025-68ba-4bf0-a92e-1f990556bfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:10.735747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a8025-68ba-4bf0-a92e-1f990556bfb7 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.736031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a8025-68ba-4bf0-a92e-1f990556bfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.736309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a8025-68ba-4bf0-a92e-1f990556bfb7 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.737666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a8025-68ba-4bf0-a92e-1f990556bfb7 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:10.739266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a8025-68ba-4bf0-a92e-1f990556bfb7 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:10.746264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 105/224] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:10 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:10.747210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a8025-68ba-4bf0-a92e-1f990556bfb7 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:10.765624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ed7d6-a2be-4553-a54f-112323b387f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:10.766220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ed7d6-a2be-4553-a54f-112323b387f0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.766499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ed7d6-a2be-4553-a54f-112323b387f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.766774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ed7d6-a2be-4553-a54f-112323b387f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.778812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ed7d6-a2be-4553-a54f-112323b387f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-670ed7d6-a2be-4553-a54f-112323b387f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:10.783703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:10.785810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 120/225] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:10.787098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ed7d6-a2be-4553-a54f-112323b387f0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:10.827142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:10.875683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:10.877280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a8025-68ba-4bf0-a92e-1f990556bfb7 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:10.901380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ed7d6-a2be-4553-a54f-112323b387f0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:10.923175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:10.924038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.924562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.925016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.925906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:10.926417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:10.928088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:10.939192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 106/226] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:10.940714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:10.945438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:10.945636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:10.946007 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:10.947021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:10.947903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] RBAC: Adding query filter params (role.id=fff136f214f2428895daf10483c542d1, scope.project.id=a667e49b60584e6382b48fec8d858ac4, user.id=9b9557fd154c48c7bb3ff6fbe036503b) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:10.948322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:10.949913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:10.956259 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 121/227] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:10 2022] GET /identity/v3/role_assignments?user.id=9b9557fd154c48c7bb3ff6fbe036503b&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=fff136f214f2428895daf10483c542d1 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:10.962559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.161676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0c29-d8be-490e-b5a8-e91910709c1c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.185041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07523fb8-1bdc-4bce-8260-650b7832df85 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.205823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58e5e-0758-4c21-a3e4-b5882d822d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.206496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58e5e-0758-4c21-a3e4-b5882d822d4d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.206777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58e5e-0758-4c21-a3e4-b5882d822d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.207053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58e5e-0758-4c21-a3e4-b5882d822d4d None admin] PATH_INFO: `/v3/users/6e154371f45e4cb081bd3c705ba1b9fb` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.208224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58e5e-0758-4c21-a3e4-b5882d822d4d None admin] RBAC: Authorizing `identity:get_user(user_id=6e154371f45e4cb081bd3c705ba1b9fb)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.209684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58e5e-0758-4c21-a3e4-b5882d822d4d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.211954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 122/228] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:10 2022] GET /identity/v3/users/6e154371f45e4cb081bd3c705ba1b9fb => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:11.217662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58e5e-0758-4c21-a3e4-b5882d822d4d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.234277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:11.248011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:11.282620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.283717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.284208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.284685 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.285721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:11.286353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.288051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.310448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 107/229] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:11.317155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.333905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58e5e-0758-4c21-a3e4-b5882d822d4d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.378523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f8934-5ec7-4417-98a9-7edb18e2a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.379181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f8934-5ec7-4417-98a9-7edb18e2a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.379466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f8934-5ec7-4417-98a9-7edb18e2a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.379742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f8934-5ec7-4417-98a9-7edb18e2a1c0 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.381619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f8934-5ec7-4417-98a9-7edb18e2a1c0 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.383113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f8934-5ec7-4417-98a9-7edb18e2a1c0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.387597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 123/230] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:11 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:11.397189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f8934-5ec7-4417-98a9-7edb18e2a1c0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.496166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73badf-0c27-48d0-9cc4-f33f87f4b1e6 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.535464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151de63c-b348-4fb7-a045-7e1e6e573d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.536491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151de63c-b348-4fb7-a045-7e1e6e573d6f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.536977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151de63c-b348-4fb7-a045-7e1e6e573d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.537454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151de63c-b348-4fb7-a045-7e1e6e573d6f None admin] PATH_INFO: `/v3/users/6e154371f45e4cb081bd3c705ba1b9fb` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.538941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151de63c-b348-4fb7-a045-7e1e6e573d6f None admin] RBAC: Authorizing `identity:get_user(user_id=6e154371f45e4cb081bd3c705ba1b9fb)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.541245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151de63c-b348-4fb7-a045-7e1e6e573d6f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.543503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 108/231] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:11 2022] GET /identity/v3/users/6e154371f45e4cb081bd3c705ba1b9fb => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:11.551664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151de63c-b348-4fb7-a045-7e1e6e573d6f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.592846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:11.641329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f8934-5ec7-4417-98a9-7edb18e2a1c0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.666263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:11.714232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151de63c-b348-4fb7-a045-7e1e6e573d6f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.722583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27754107-1bf1-4799-9801-abd22cf8624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.723501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27754107-1bf1-4799-9801-abd22cf8624b None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.723684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27754107-1bf1-4799-9801-abd22cf8624b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.723934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27754107-1bf1-4799-9801-abd22cf8624b None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4/users/6e154371f45e4cb081bd3c705ba1b9fb/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.725992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27754107-1bf1-4799-9801-abd22cf8624b None admin] RBAC: Authorizing `identity:create_grant(project_id=a667e49b60584e6382b48fec8d858ac4, user_id=6e154371f45e4cb081bd3c705ba1b9fb, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.728126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27754107-1bf1-4799-9801-abd22cf8624b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.754528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dc95f-9c04-4739-a4a3-783a3dbe4440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.755562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dc95f-9c04-4739-a4a3-783a3dbe4440 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.756016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dc95f-9c04-4739-a4a3-783a3dbe4440 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.756493 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dc95f-9c04-4739-a4a3-783a3dbe4440 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.757876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dc95f-9c04-4739-a4a3-783a3dbe4440 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.760464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dc95f-9c04-4739-a4a3-783a3dbe4440 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.762832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 109/232] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:11 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:11.769196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dc95f-9c04-4739-a4a3-783a3dbe4440 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.828994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 124/233] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:11 2022] PUT /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4/users/6e154371f45e4cb081bd3c705ba1b9fb/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:11.905102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dc95f-9c04-4739-a4a3-783a3dbe4440 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.930515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:11.952561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d2dc1-76e7-461f-a3c4-0feda1c0274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.953265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d2dc1-76e7-461f-a3c4-0feda1c0274e None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.953595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d2dc1-76e7-461f-a3c4-0feda1c0274e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.953896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d2dc1-76e7-461f-a3c4-0feda1c0274e None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4/users/6e154371f45e4cb081bd3c705ba1b9fb/roles/d6fe1aca91e74aad9907c855501f588f` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.959985 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d2dc1-76e7-461f-a3c4-0feda1c0274e None admin] RBAC: Authorizing `identity:create_grant(project_id=a667e49b60584e6382b48fec8d858ac4, user_id=6e154371f45e4cb081bd3c705ba1b9fb, role_id=d6fe1aca91e74aad9907c855501f588f)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.962169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d2dc1-76e7-461f-a3c4-0feda1c0274e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:12.103011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 110/234] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:11 2022] PUT /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4/users/6e154371f45e4cb081bd3c705ba1b9fb/roles/d6fe1aca91e74aad9907c855501f588f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:12.607167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d56d6-169e-4e52-b3ab-2b78cdf66626 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:12.607700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d56d6-169e-4e52-b3ab-2b78cdf66626 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:12.607700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d56d6-169e-4e52-b3ab-2b78cdf66626 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:12.609374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 125/235] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:12.614741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f52765-15c7-44c7-95d3-51b9dec0ed22 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:12.615200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f52765-15c7-44c7-95d3-51b9dec0ed22 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:12.615276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f52765-15c7-44c7-95d3-51b9dec0ed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:12.642943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f2f52765-15c7-44c7-95d3-51b9dec0ed22 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:12.644498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f2f52765-15c7-44c7-95d3-51b9dec0ed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:12.735538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 111/236] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:12.795188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6af60-25e6-459d-a024-6ee677fbed74 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:12.795795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6af60-25e6-459d-a024-6ee677fbed74 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:12.796307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6af60-25e6-459d-a024-6ee677fbed74 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:12.797890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 126/237] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:12.803420 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c340-b6fe-4dd3-a4bb-d1531316a227 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:12.803940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c340-b6fe-4dd3-a4bb-d1531316a227 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:12.804421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c340-b6fe-4dd3-a4bb-d1531316a227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:12.834911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ce08c340-b6fe-4dd3-a4bb-d1531316a227 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:12.836466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ce08c340-b6fe-4dd3-a4bb-d1531316a227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:12.955483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 112/238] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:13.131217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9bc48-4509-4f52-b6e3-a1b25bccdbe9 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:13.131882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9bc48-4509-4f52-b6e3-a1b25bccdbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:13.132378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9bc48-4509-4f52-b6e3-a1b25bccdbe9 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:13.134911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 127/239] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:13.140324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77178373-d1e5-40f7-954c-050443a01c1d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:13.140949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77178373-d1e5-40f7-954c-050443a01c1d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:13.141511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77178373-d1e5-40f7-954c-050443a01c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:13.168719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-77178373-d1e5-40f7-954c-050443a01c1d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:13.170319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-77178373-d1e5-40f7-954c-050443a01c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:13.236856 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:13.282963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9bc48-4509-4f52-b6e3-a1b25bccdbe9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:13.289912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 113/240] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:13.339378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77178373-d1e5-40f7-954c-050443a01c1d None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:13.464434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77178373-d1e5-40f7-954c-050443a01c1d None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:13.495548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9bc48-4509-4f52-b6e3-a1b25bccdbe9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:13.536434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfb42f-6a30-4e5f-808d-d3689419074b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:13.537083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfb42f-6a30-4e5f-808d-d3689419074b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:13.537361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfb42f-6a30-4e5f-808d-d3689419074b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:13.537623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfb42f-6a30-4e5f-808d-d3689419074b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:13.562504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfb42f-6a30-4e5f-808d-d3689419074b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-42dfb42f-6a30-4e5f-808d-d3689419074b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:13.570014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:13.571041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:13.572261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 114/241] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:13.578217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfb42f-6a30-4e5f-808d-d3689419074b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:13.580156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0777a-8657-4e0f-890a-aa37f75c2cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:13.580828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0777a-8657-4e0f-890a-aa37f75c2cbd None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:13.581113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0777a-8657-4e0f-890a-aa37f75c2cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:13.581481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0777a-8657-4e0f-890a-aa37f75c2cbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:13.582839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0777a-8657-4e0f-890a-aa37f75c2cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:13.584358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0777a-8657-4e0f-890a-aa37f75c2cbd None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:13.587064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 128/242] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:13.593311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0777a-8657-4e0f-890a-aa37f75c2cbd None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:13.600717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:13.628737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:13.719478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfb42f-6a30-4e5f-808d-d3689419074b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:13.735976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0777a-8657-4e0f-890a-aa37f75c2cbd None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:13.769334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:13.770349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:13.770768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:13.771171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:13.771970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:13.772425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:13.774475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:13.774747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349451ad-fa30-4f50-b72f-df48ebc8804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:13.775462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349451ad-fa30-4f50-b72f-df48ebc8804d None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:13.775842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349451ad-fa30-4f50-b72f-df48ebc8804d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:13.776214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349451ad-fa30-4f50-b72f-df48ebc8804d None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:13.776965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349451ad-fa30-4f50-b72f-df48ebc8804d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:13.778533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349451ad-fa30-4f50-b72f-df48ebc8804d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:13.784644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 115/243] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:13.785765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:13.865937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 129/244] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:13 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:13.867076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349451ad-fa30-4f50-b72f-df48ebc8804d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:13.933269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:13.949236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b245e1-3e9d-4e63-846a-143e2c523bb4 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:13.992144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.041752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349451ad-fa30-4f50-b72f-df48ebc8804d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.048543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58013d-e675-4507-9f93-0ca4e26251e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.049360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58013d-e675-4507-9f93-0ca4e26251e0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.049742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58013d-e675-4507-9f93-0ca4e26251e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.050143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58013d-e675-4507-9f93-0ca4e26251e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.063978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58013d-e675-4507-9f93-0ca4e26251e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-ac58013d-e675-4507-9f93-0ca4e26251e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:14.069619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:14.070532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:14.072055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 116/245] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:14.078457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58013d-e675-4507-9f93-0ca4e26251e0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.091999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40978730-45a9-42bd-b007-ddb805e01f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.092656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40978730-45a9-42bd-b007-ddb805e01f93 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.092937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40978730-45a9-42bd-b007-ddb805e01f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.093212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40978730-45a9-42bd-b007-ddb805e01f93 None admin] PATH_INFO: `/v3/users/02fd5af397ee4fd79406fd4017635d8e` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.111767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40978730-45a9-42bd-b007-ddb805e01f93 None admin] RBAC: Authorizing `identity:get_user(user_id=02fd5af397ee4fd79406fd4017635d8e)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.113313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40978730-45a9-42bd-b007-ddb805e01f93 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.116305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 130/246] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:13 2022] GET /identity/v3/users/02fd5af397ee4fd79406fd4017635d8e => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:14.127280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40978730-45a9-42bd-b007-ddb805e01f93 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.219824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58013d-e675-4507-9f93-0ca4e26251e0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.260978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40978730-45a9-42bd-b007-ddb805e01f93 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.264307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4c488-9c78-435c-8141-0971e461d87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.265233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4c488-9c78-435c-8141-0971e461d87d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.266079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4c488-9c78-435c-8141-0971e461d87d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.266579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4c488-9c78-435c-8141-0971e461d87d None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.267471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4c488-9c78-435c-8141-0971e461d87d None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:14.267997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4c488-9c78-435c-8141-0971e461d87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.269570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4c488-9c78-435c-8141-0971e461d87d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.280560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 117/247] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:14.282235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb6be3-12f3-4299-868d-66673b077d00 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.282791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb6be3-12f3-4299-868d-66673b077d00 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.283281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb6be3-12f3-4299-868d-66673b077d00 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.284731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 118/248] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:14.287991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb6be3-12f3-4299-868d-66673b077d00 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.295505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.309438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.329174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e8218-a3af-4f76-8946-0f08b06fd30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.329174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e8218-a3af-4f76-8946-0f08b06fd30e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.329174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e8218-a3af-4f76-8946-0f08b06fd30e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.330632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e8218-a3af-4f76-8946-0f08b06fd30e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.423094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb6be3-12f3-4299-868d-66673b077d00 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.451827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e8218-a3af-4f76-8946-0f08b06fd30e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.453307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e8218-a3af-4f76-8946-0f08b06fd30e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.482258 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaf7cb-581e-4338-9435-2a4122df1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.482904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf7cb-581e-4338-9435-2a4122df1b24 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.483182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf7cb-581e-4338-9435-2a4122df1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.483457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf7cb-581e-4338-9435-2a4122df1b24 None admin] PATH_INFO: `/v3/users/9b9557fd154c48c7bb3ff6fbe036503b` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.486055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdaf7cb-581e-4338-9435-2a4122df1b24 None admin] RBAC: Authorizing `identity:get_user(user_id=9b9557fd154c48c7bb3ff6fbe036503b)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.487529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdaf7cb-581e-4338-9435-2a4122df1b24 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.489504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 119/249] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/users/9b9557fd154c48c7bb3ff6fbe036503b => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:14.490721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3f914-2943-48a6-b83e-9ae408dea842 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.491001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3f914-2943-48a6-b83e-9ae408dea842 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.491271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3f914-2943-48a6-b83e-9ae408dea842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.511779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1ad3f914-2943-48a6-b83e-9ae408dea842 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:14.512909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1ad3f914-2943-48a6-b83e-9ae408dea842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:14.585878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 120/250] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:14.587348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3f914-2943-48a6-b83e-9ae408dea842 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.603934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.703025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3f914-2943-48a6-b83e-9ae408dea842 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.706701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 131/251] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:14.708251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00acda-b1f4-4144-bfb6-5f32a1c7cc37 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.708734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00acda-b1f4-4144-bfb6-5f32a1c7cc37 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.709223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00acda-b1f4-4144-bfb6-5f32a1c7cc37 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.715005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 132/252] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:14.722248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40078f-7df5-4fb9-bf60-cd74b4e8b5e9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.722834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40078f-7df5-4fb9-bf60-cd74b4e8b5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.723301 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40078f-7df5-4fb9-bf60-cd74b4e8b5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.750766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085a836-2a5c-494b-8d30-c158382fbb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.751457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a836-2a5c-494b-8d30-c158382fbb18 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.751741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a836-2a5c-494b-8d30-c158382fbb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.752018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a836-2a5c-494b-8d30-c158382fbb18 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.753295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085a836-2a5c-494b-8d30-c158382fbb18 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.755129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085a836-2a5c-494b-8d30-c158382fbb18 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.756836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 121/253] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:14.757626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085a836-2a5c-494b-8d30-c158382fbb18 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.762356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-4b40078f-7df5-4fb9-bf60-cd74b4e8b5e9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:14.763467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4b40078f-7df5-4fb9-bf60-cd74b4e8b5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:14.790599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.917757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 133/254] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:14.920934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40078f-7df5-4fb9-bf60-cd74b4e8b5e9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.927213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.960435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085a836-2a5c-494b-8d30-c158382fbb18 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:15.026983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44d3e-e99a-4480-90fb-dddcb6431c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:15.027579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44d3e-e99a-4480-90fb-dddcb6431c19 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:15.027854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44d3e-e99a-4480-90fb-dddcb6431c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:15.028226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44d3e-e99a-4480-90fb-dddcb6431c19 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:15.030561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee44d3e-e99a-4480-90fb-dddcb6431c19 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:15.032331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee44d3e-e99a-4480-90fb-dddcb6431c19 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:15.034423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 122/255] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:15.035598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44d3e-e99a-4480-90fb-dddcb6431c19 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:15.158889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40078f-7df5-4fb9-bf60-cd74b4e8b5e9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:15.262861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbedc38-30e7-4eb2-b880-3c8319fe3fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:15.264081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbedc38-30e7-4eb2-b880-3c8319fe3fed None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:15.264631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbedc38-30e7-4eb2-b880-3c8319fe3fed None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:15.265096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbedc38-30e7-4eb2-b880-3c8319fe3fed None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4/users/9b9557fd154c48c7bb3ff6fbe036503b/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:15.269457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbedc38-30e7-4eb2-b880-3c8319fe3fed None admin] RBAC: Authorizing `identity:create_grant(project_id=a667e49b60584e6382b48fec8d858ac4, user_id=9b9557fd154c48c7bb3ff6fbe036503b, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:15.271225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbedc38-30e7-4eb2-b880-3c8319fe3fed None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:15.283910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44d3e-e99a-4480-90fb-dddcb6431c19 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:15.329277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2e7c9-23c6-44f0-8fb2-026ca5698d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:15.330364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2e7c9-23c6-44f0-8fb2-026ca5698d0b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:15.330899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2e7c9-23c6-44f0-8fb2-026ca5698d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:15.331368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2e7c9-23c6-44f0-8fb2-026ca5698d0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:15.342195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 134/256] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:14 2022] PUT /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4/users/9b9557fd154c48c7bb3ff6fbe036503b/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:15.350046 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2e7c9-23c6-44f0-8fb2-026ca5698d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:15.350664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbedc38-30e7-4eb2-b880-3c8319fe3fed None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-faf2e7c9-23c6-44f0-8fb2-026ca5698d0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:15.352778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:15.353697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:15.354517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 123/257] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:15.355245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2e7c9-23c6-44f0-8fb2-026ca5698d0b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:15.369223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:15.382528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:15.516508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbedc38-30e7-4eb2-b880-3c8319fe3fed None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:15.536175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2e7c9-23c6-44f0-8fb2-026ca5698d0b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:15.559276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:15.559949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:15.560820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:15.561105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:15.561895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] RBAC: Adding query filter params (role.id=f9e4fcd3595c43609a8c949b1996b02c, scope.project.id=a667e49b60584e6382b48fec8d858ac4, user.id=02fd5af397ee4fd79406fd4017635d8e) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:15.562215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:15.563703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:15.571552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 135/258] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:15 2022] GET /identity/v3/role_assignments?user.id=02fd5af397ee4fd79406fd4017635d8e&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=f9e4fcd3595c43609a8c949b1996b02c => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:15.572463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:15.627673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb7c3e-254b-4804-8841-4bdae475a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:15.628795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb7c3e-254b-4804-8841-4bdae475a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:15.629284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb7c3e-254b-4804-8841-4bdae475a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:15.629759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb7c3e-254b-4804-8841-4bdae475a5c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:15.661909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb7c3e-254b-4804-8841-4bdae475a5c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-97eb7c3e-254b-4804-8841-4bdae475a5c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:15.668061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:15.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:15.669818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:15.675633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:15.677893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 124/259] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:15.687223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb7c3e-254b-4804-8841-4bdae475a5c3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:15.715068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c9a6a-54ab-4a69-b882-57d22c7aec14 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:15.763175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:15.764084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:15.764605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:15.765062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:15.766093 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:15.766524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:15.770456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:15.781234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 136/260] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:15.787193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:15.836900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:15.912467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb7c3e-254b-4804-8841-4bdae475a5c3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:15.935369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b6f58-50a1-4120-8987-a60b487f8855 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:15.987730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:15.988603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:15.989816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:15.991024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:15.992919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:15.993414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:15.995542 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:16.013056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 125/261] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:16.019279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbf015-8cd5-438a-a3bc-3997bdefb500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:16.020157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbf015-8cd5-438a-a3bc-3997bdefb500 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.020625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbf015-8cd5-438a-a3bc-3997bdefb500 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.021108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbf015-8cd5-438a-a3bc-3997bdefb500 None admin] PATH_INFO: `/v3/users/6e154371f45e4cb081bd3c705ba1b9fb` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.021359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:16.026680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbf015-8cd5-438a-a3bc-3997bdefb500 None admin] RBAC: Authorizing `identity:get_user(user_id=6e154371f45e4cb081bd3c705ba1b9fb)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:16.028977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbf015-8cd5-438a-a3bc-3997bdefb500 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:16.032055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 137/262] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:15 2022] GET /identity/v3/users/6e154371f45e4cb081bd3c705ba1b9fb => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:16.038825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbf015-8cd5-438a-a3bc-3997bdefb500 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:16.046215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:16.218180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbf015-8cd5-438a-a3bc-3997bdefb500 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:16.274642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81643a2-eac5-4aa1-bf75-283a055c7fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:16.275281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81643a2-eac5-4aa1-bf75-283a055c7fad None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.275614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81643a2-eac5-4aa1-bf75-283a055c7fad None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.275897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81643a2-eac5-4aa1-bf75-283a055c7fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.303337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353485ba-1eeb-4548-82be-48c691fb723e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:16.372619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:16.387522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfebe3-23e9-4fc5-82b3-83d324749eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:16.388180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfebe3-23e9-4fc5-82b3-83d324749eed None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.388464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfebe3-23e9-4fc5-82b3-83d324749eed None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.392198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfebe3-23e9-4fc5-82b3-83d324749eed None admin] PATH_INFO: `/v3/users/6e154371f45e4cb081bd3c705ba1b9fb` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.394675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfebe3-23e9-4fc5-82b3-83d324749eed None admin] RBAC: Authorizing `identity:get_user(user_id=6e154371f45e4cb081bd3c705ba1b9fb)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:16.396179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfebe3-23e9-4fc5-82b3-83d324749eed None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:16.398220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 126/263] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:16 2022] GET /identity/v3/users/6e154371f45e4cb081bd3c705ba1b9fb => generated 283 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:16.404071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfebe3-23e9-4fc5-82b3-83d324749eed None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:16.416901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81643a2-eac5-4aa1-bf75-283a055c7fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:16.422074 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81643a2-eac5-4aa1-bf75-283a055c7fad None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:16.425155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:16.630449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfebe3-23e9-4fc5-82b3-83d324749eed None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:16.681781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 138/264] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:29:16.687672 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81643a2-eac5-4aa1-bf75-283a055c7fad None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:16.689563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7bee3-a0ac-46f4-af73-e10da9b92f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:16.690727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7bee3-a0ac-46f4-af73-e10da9b92f01 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.691010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7bee3-a0ac-46f4-af73-e10da9b92f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.691284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7bee3-a0ac-46f4-af73-e10da9b92f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.824424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:16.829063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7bee3-a0ac-46f4-af73-e10da9b92f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:16.830609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7bee3-a0ac-46f4-af73-e10da9b92f01 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:16.867844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81643a2-eac5-4aa1-bf75-283a055c7fad None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:16.870632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:16.918288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887953d4-96c9-4eca-a31d-65041207c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:16.918943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887953d4-96c9-4eca-a31d-65041207c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.919218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887953d4-96c9-4eca-a31d-65041207c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.919487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887953d4-96c9-4eca-a31d-65041207c8e6 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.921912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887953d4-96c9-4eca-a31d-65041207c8e6 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:16.923360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887953d4-96c9-4eca-a31d-65041207c8e6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:16.929956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 139/265] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:16 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:16.929956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3f536-eb95-4373-b147-da67e7a31c5b None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.929956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3f536-eb95-4373-b147-da67e7a31c5b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.929956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3f536-eb95-4373-b147-da67e7a31c5b None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.929956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 140/266] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:16.929956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ddb96-d7ec-4551-9962-4e2f31a59b7e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.930397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ddb96-d7ec-4551-9962-4e2f31a59b7e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.930397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ddb96-d7ec-4551-9962-4e2f31a59b7e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.932172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 141/267] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:16.933185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ddb96-d7ec-4551-9962-4e2f31a59b7e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:16.990244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 127/268] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:29:16.992192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f78e0-4f93-4770-ad27-47445e052a10 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.992700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f78e0-4f93-4770-ad27-47445e052a10 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.993186 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f78e0-4f93-4770-ad27-47445e052a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.015420 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-2c7f78e0-4f93-4770-ad27-47445e052a10 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:17.017217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2c7f78e0-4f93-4770-ad27-47445e052a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:17.067139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ddb96-d7ec-4551-9962-4e2f31a59b7e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:17.097190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 128/269] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:17.099205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b3d4-8ce9-45d1-8a62-e16daabdaf2c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:17.099721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b3d4-8ce9-45d1-8a62-e16daabdaf2c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:17.100193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b3d4-8ce9-45d1-8a62-e16daabdaf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.106247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:17.106841 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:17.107123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:17.107395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.108158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=a667e49b60584e6382b48fec8d858ac4, user.id=6e154371f45e4cb081bd3c705ba1b9fb) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:17.108454 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:17.109985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:17.116699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 142/270] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:16 2022] GET /identity/v3/role_assignments?user.id=6e154371f45e4cb081bd3c705ba1b9fb&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:17.122035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:17.144376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-8e06b3d4-8ce9-45d1-8a62-e16daabdaf2c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:17.145658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8e06b3d4-8ce9-45d1-8a62-e16daabdaf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:17.206100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:17.208726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:17.303403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 129/271] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:17.327844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1366c0-c82a-4ddf-a06a-9b4e452f08d9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:17.364759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2f78a-ac57-4554-bca1-c789487b69dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:17.365405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f78a-ac57-4554-bca1-c789487b69dc None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:17.365686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f78a-ac57-4554-bca1-c789487b69dc None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:17.366000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f78a-ac57-4554-bca1-c789487b69dc None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.367197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2f78a-ac57-4554-bca1-c789487b69dc None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:17.369255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2f78a-ac57-4554-bca1-c789487b69dc None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:17.371176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 143/272] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:17 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:17.377147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06b3d4-8ce9-45d1-8a62-e16daabdaf2c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:17.507939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06b3d4-8ce9-45d1-8a62-e16daabdaf2c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:17.544182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:17.544796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:17.545078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:17.545395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.546169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] RBAC: Adding query filter params (role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=a667e49b60584e6382b48fec8d858ac4, user.id=6e154371f45e4cb081bd3c705ba1b9fb) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:17.546476 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:17.547926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:17.554368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 130/273] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:17 2022] GET /identity/v3/role_assignments?user.id=6e154371f45e4cb081bd3c705ba1b9fb&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=d6fe1aca91e74aad9907c855501f588f => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:17.721357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2f78a-ac57-4554-bca1-c789487b69dc None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:17.764277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:17.798688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:17.843254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2f78a-ac57-4554-bca1-c789487b69dc None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:17.844566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:17.884207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a85c7-07e4-4cbe-acf5-69efa7a14634 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:17.886373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d76b0-5048-4d91-9f56-92063cdb1fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:17.887023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d76b0-5048-4d91-9f56-92063cdb1fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:17.887305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d76b0-5048-4d91-9f56-92063cdb1fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:17.887579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d76b0-5048-4d91-9f56-92063cdb1fd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.889206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d76b0-5048-4d91-9f56-92063cdb1fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:17.890527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d76b0-5048-4d91-9f56-92063cdb1fd0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:17.892460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 144/274] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:17.909048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d76b0-5048-4d91-9f56-92063cdb1fd0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:17.951045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0493871-ab28-4668-9f85-ccc2d6405b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:17.951725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0493871-ab28-4668-9f85-ccc2d6405b18 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:17.951969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0493871-ab28-4668-9f85-ccc2d6405b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:17.952242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0493871-ab28-4668-9f85-ccc2d6405b18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.965941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0493871-ab28-4668-9f85-ccc2d6405b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-d0493871-ab28-4668-9f85-ccc2d6405b18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:17.971730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:17.972714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:17.973525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:17.975469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 131/275] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:17.982038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0493871-ab28-4668-9f85-ccc2d6405b18 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:18.067453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d76b0-5048-4d91-9f56-92063cdb1fd0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:18.095069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:18.117276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eeba5e-328b-4017-822c-1d1179814d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:18.119253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeba5e-328b-4017-822c-1d1179814d95 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.119689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeba5e-328b-4017-822c-1d1179814d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.120071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeba5e-328b-4017-822c-1d1179814d95 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.120793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0493871-ab28-4668-9f85-ccc2d6405b18 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:18.120971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eeba5e-328b-4017-822c-1d1179814d95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:18.122581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eeba5e-328b-4017-822c-1d1179814d95 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:18.160568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecf941-f086-42b5-902b-241eacfaf00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:18.161179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf941-f086-42b5-902b-241eacfaf00b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.161462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf941-f086-42b5-902b-241eacfaf00b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.161736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf941-f086-42b5-902b-241eacfaf00b None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.162443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecf941-f086-42b5-902b-241eacfaf00b None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:18.162763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecf941-f086-42b5-902b-241eacfaf00b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:18.164199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecf941-f086-42b5-902b-241eacfaf00b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:18.173472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 132/276] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:18.175870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d299-f8ad-470a-aa64-331ad4859453 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.176158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d299-f8ad-470a-aa64-331ad4859453 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.176435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d299-f8ad-470a-aa64-331ad4859453 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.177885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 133/277] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:18.179098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255d299-f8ad-470a-aa64-331ad4859453 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:18.227518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 145/278] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:17 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:29:18.235390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380201b3-2b7a-4b33-97bd-f6cc6f504189 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.235680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380201b3-2b7a-4b33-97bd-f6cc6f504189 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.235962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380201b3-2b7a-4b33-97bd-f6cc6f504189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.273191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-380201b3-2b7a-4b33-97bd-f6cc6f504189 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:18.274625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-380201b3-2b7a-4b33-97bd-f6cc6f504189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:18.295645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255d299-f8ad-470a-aa64-331ad4859453 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:18.298133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:18.342532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921055b6-15d5-413d-aad1-25215f04cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:18.343102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921055b6-15d5-413d-aad1-25215f04cb6d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.343387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921055b6-15d5-413d-aad1-25215f04cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.343732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921055b6-15d5-413d-aad1-25215f04cb6d None admin] PATH_INFO: `/v3/users/73e1d4589ca144449e971100b5d65917` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.352781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 146/279] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:18.362500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921055b6-15d5-413d-aad1-25215f04cb6d None admin] RBAC: Authorizing `identity:get_user(user_id=73e1d4589ca144449e971100b5d65917)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:18.364603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921055b6-15d5-413d-aad1-25215f04cb6d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:18.369907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 134/280] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:18 2022] GET /identity/v3/users/73e1d4589ca144449e971100b5d65917 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:18.370134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb1d0f-c99a-41fb-8126-3e97fcb73664 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.370134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb1d0f-c99a-41fb-8126-3e97fcb73664 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.370134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb1d0f-c99a-41fb-8126-3e97fcb73664 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.375100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 147/281] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:18 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:18.377335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef130352-0cf4-416c-9b0e-dc94e5c6752b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.377629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef130352-0cf4-416c-9b0e-dc94e5c6752b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.377937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef130352-0cf4-416c-9b0e-dc94e5c6752b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.380268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb1d0f-c99a-41fb-8126-3e97fcb73664 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:18.386347 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:18.400104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ef130352-0cf4-416c-9b0e-dc94e5c6752b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:18.401359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ef130352-0cf4-416c-9b0e-dc94e5c6752b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:18.488503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 135/282] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:18.515262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb1d0f-c99a-41fb-8126-3e97fcb73664 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:18.561388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:18.562391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.562878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.563400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.564380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] RBAC: Adding query filter params (role.id=fff136f214f2428895daf10483c542d1, user.id=73e1d4589ca144449e971100b5d65917, scope.system=all) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:18.564900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:18.566875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:18.573355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 148/283] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 24 17:29:18 2022] GET /identity/v3/role_assignments?user.id=73e1d4589ca144449e971100b5d65917&scope.system=all&role.id=fff136f214f2428895daf10483c542d1 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:18.628263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef130352-0cf4-416c-9b0e-dc94e5c6752b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:18.760553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef130352-0cf4-416c-9b0e-dc94e5c6752b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:18.786214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:18.796966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:18.808154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3730ae-2fe5-44de-b8dc-9c0c72bcafe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:18.808916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3730ae-2fe5-44de-b8dc-9c0c72bcafe5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:18.817145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3730ae-2fe5-44de-b8dc-9c0c72bcafe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:18.817145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3730ae-2fe5-44de-b8dc-9c0c72bcafe5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:18.817339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:18.830891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3730ae-2fe5-44de-b8dc-9c0c72bcafe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-ee3730ae-2fe5-44de-b8dc-9c0c72bcafe5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:18.835677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:18.837179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:18.837989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:18.842224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 136/284] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:18.843634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3730ae-2fe5-44de-b8dc-9c0c72bcafe5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:18.965079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3730ae-2fe5-44de-b8dc-9c0c72bcafe5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:19.002120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:19.003090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.003535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.003957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.004781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:19.005284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:19.006972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:19.016863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 137/285] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:19.023580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:19.028141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac69058-af5e-4d9d-bc03-f1c3d093a23e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:19.095470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b761ae-3352-4431-a910-2ab979b3f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:19.096135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b761ae-3352-4431-a910-2ab979b3f675 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.096420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b761ae-3352-4431-a910-2ab979b3f675 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.096699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b761ae-3352-4431-a910-2ab979b3f675 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.116923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b761ae-3352-4431-a910-2ab979b3f675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-b7b761ae-3352-4431-a910-2ab979b3f675 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:19.129539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:19.130546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:19.131339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:19.135704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 149/286] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:19.138934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b761ae-3352-4431-a910-2ab979b3f675 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:19.139540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:19.229342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f3b61-d0f5-42b4-9569-0fcaa232f6a7 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:19.301742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0569cc1-09ca-46bd-a9bd-c539eebb6ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:19.302559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0569cc1-09ca-46bd-a9bd-c539eebb6ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.302846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0569cc1-09ca-46bd-a9bd-c539eebb6ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.303146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0569cc1-09ca-46bd-a9bd-c539eebb6ac0 None admin] PATH_INFO: `/v3/users/02fd5af397ee4fd79406fd4017635d8e` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.304349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0569cc1-09ca-46bd-a9bd-c539eebb6ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=02fd5af397ee4fd79406fd4017635d8e)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:19.306449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0569cc1-09ca-46bd-a9bd-c539eebb6ac0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:19.307661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:19.309311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 138/287] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:19 2022] GET /identity/v3/users/02fd5af397ee4fd79406fd4017635d8e => generated 297 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:19.325594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0569cc1-09ca-46bd-a9bd-c539eebb6ac0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:19.348809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b761ae-3352-4431-a910-2ab979b3f675 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:19.407818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:19.408903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.409399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.409896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.411135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:19.411744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:19.416705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:19.428769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 150/288] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:19.436011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:19.576693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5dcca-df29-4d33-a585-aad20f53fed2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:19.592116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0569cc1-09ca-46bd-a9bd-c539eebb6ac0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:19.628767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432274ba-10de-4b0a-9d04-c74f63652fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:19.630849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432274ba-10de-4b0a-9d04-c74f63652fce None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.634395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432274ba-10de-4b0a-9d04-c74f63652fce None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.634939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432274ba-10de-4b0a-9d04-c74f63652fce None admin] PATH_INFO: `/v3/users/9b9557fd154c48c7bb3ff6fbe036503b` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.637009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432274ba-10de-4b0a-9d04-c74f63652fce None admin] RBAC: Authorizing `identity:get_user(user_id=9b9557fd154c48c7bb3ff6fbe036503b)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:19.638779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432274ba-10de-4b0a-9d04-c74f63652fce None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:19.644744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:19.652271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 151/289] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:19 2022] GET /identity/v3/users/9b9557fd154c48c7bb3ff6fbe036503b => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:19.654209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432274ba-10de-4b0a-9d04-c74f63652fce None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:19.671874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:19.693297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5e0d8-130d-4060-9138-bcf32db47151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:19.694202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e0d8-130d-4060-9138-bcf32db47151 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.694519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e0d8-130d-4060-9138-bcf32db47151 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.694810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e0d8-130d-4060-9138-bcf32db47151 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.697260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5e0d8-130d-4060-9138-bcf32db47151 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:19.698987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5e0d8-130d-4060-9138-bcf32db47151 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:19.701197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 139/290] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:19 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:19.709682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5e0d8-130d-4060-9138-bcf32db47151 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:19.858532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432274ba-10de-4b0a-9d04-c74f63652fce None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:19.942417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bf479-71c0-4f3b-b2d0-cdf0a132c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:19.943056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bf479-71c0-4f3b-b2d0-cdf0a132c620 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.943342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bf479-71c0-4f3b-b2d0-cdf0a132c620 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.943619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bf479-71c0-4f3b-b2d0-cdf0a132c620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.959996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5e0d8-130d-4060-9138-bcf32db47151 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:20.042584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c64fa-db73-4668-b38a-4b84fe27bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:20.043016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47bf479-71c0-4f3b-b2d0-cdf0a132c620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:20.043448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c64fa-db73-4668-b38a-4b84fe27bf59 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:20.043607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c64fa-db73-4668-b38a-4b84fe27bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:20.043835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c64fa-db73-4668-b38a-4b84fe27bf59 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4/users/02fd5af397ee4fd79406fd4017635d8e/roles/f9e4fcd3595c43609a8c949b1996b02c` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:20.044527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47bf479-71c0-4f3b-b2d0-cdf0a132c620 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:20.045988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c64fa-db73-4668-b38a-4b84fe27bf59 None admin] RBAC: Authorizing `identity:create_grant(project_id=a667e49b60584e6382b48fec8d858ac4, user_id=02fd5af397ee4fd79406fd4017635d8e, role_id=f9e4fcd3595c43609a8c949b1996b02c)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:20.047763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c64fa-db73-4668-b38a-4b84fe27bf59 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:20.063946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:20.145312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 140/291] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:19 2022] PUT /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4/users/02fd5af397ee4fd79406fd4017635d8e/roles/f9e4fcd3595c43609a8c949b1996b02c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:20.164181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 152/292] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:20.175207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c64fa-db73-4668-b38a-4b84fe27bf59 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:20.270521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:20.338762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c64fa-db73-4668-b38a-4b84fe27bf59 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:20.379107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2ac7-e7ea-4bee-ab92-028ca2485b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:20.380230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2ac7-e7ea-4bee-ab92-028ca2485b9e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:20.380789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2ac7-e7ea-4bee-ab92-028ca2485b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:20.381319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2ac7-e7ea-4bee-ab92-028ca2485b9e None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:20.382851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2ac7-e7ea-4bee-ab92-028ca2485b9e None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:20.384761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2ac7-e7ea-4bee-ab92-028ca2485b9e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:20.387412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 141/293] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:20 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:20.393338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bf479-71c0-4f3b-b2d0-cdf0a132c620 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:20.542645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bf479-71c0-4f3b-b2d0-cdf0a132c620 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:20.559368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:20.595431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f13acc-4d2e-4c8a-be7e-468e63b5025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:20.596486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13acc-4d2e-4c8a-be7e-468e63b5025a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:20.596977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13acc-4d2e-4c8a-be7e-468e63b5025a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:20.597561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13acc-4d2e-4c8a-be7e-468e63b5025a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:20.598619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f13acc-4d2e-4c8a-be7e-468e63b5025a None admin] RBAC: Adding query filter params (role.id=fff136f214f2428895daf10483c542d1, scope.project.id=a667e49b60584e6382b48fec8d858ac4, user.id=9b9557fd154c48c7bb3ff6fbe036503b) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:20.599145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f13acc-4d2e-4c8a-be7e-468e63b5025a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:20.600922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f13acc-4d2e-4c8a-be7e-468e63b5025a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:20.608805 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 153/294] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:20 2022] GET /identity/v3/role_assignments?user.id=9b9557fd154c48c7bb3ff6fbe036503b&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=fff136f214f2428895daf10483c542d1 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:20.947875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea778f-dbc9-4467-bd41-bbe830708e19 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:20.948508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea778f-dbc9-4467-bd41-bbe830708e19 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:20.948981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea778f-dbc9-4467-bd41-bbe830708e19 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:20.951121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 142/295] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:29:20.959243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23770e3-e74f-465a-a236-3f39a49d9e4c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:20.959243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23770e3-e74f-465a-a236-3f39a49d9e4c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:20.959243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23770e3-e74f-465a-a236-3f39a49d9e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:20.988026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f23770e3-e74f-465a-a236-3f39a49d9e4c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:20.989575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f23770e3-e74f-465a-a236-3f39a49d9e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:21.003521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78d9d-e818-4909-b6fb-031240ef1624 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.004137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78d9d-e818-4909-b6fb-031240ef1624 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.004616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78d9d-e818-4909-b6fb-031240ef1624 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.006253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 143/296] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:29:21.012780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c22ff-c012-4c02-8758-59da0ae42c51 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.013377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c22ff-c012-4c02-8758-59da0ae42c51 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.013929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c22ff-c012-4c02-8758-59da0ae42c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.038506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7b7c22ff-c012-4c02-8758-59da0ae42c51 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:21.040562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7b7c22ff-c012-4c02-8758-59da0ae42c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:21.107658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 154/297] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:21.138782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 144/298] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:21.165117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70691-e682-49aa-a620-914d868bc72f None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.165779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70691-e682-49aa-a620-914d868bc72f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.166280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70691-e682-49aa-a620-914d868bc72f None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.168286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 155/299] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:21.176141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158881-10db-4c71-aff2-78a655233e63 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.176607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158881-10db-4c71-aff2-78a655233e63 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.176813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158881-10db-4c71-aff2-78a655233e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.196050 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c3158881-10db-4c71-aff2-78a655233e63 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:21.197441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c3158881-10db-4c71-aff2-78a655233e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:21.251888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:21.278623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 145/300] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:21.434188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f70691-e682-49aa-a620-914d868bc72f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:21.434608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3158881-10db-4c71-aff2-78a655233e63 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:21.566872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3158881-10db-4c71-aff2-78a655233e63 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:21.575859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f70691-e682-49aa-a620-914d868bc72f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:21.580904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:21.609794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcccd9c-7fc2-4bdb-b409-aef8668e57ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:21.610029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcccd9c-7fc2-4bdb-b409-aef8668e57ec None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.610279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcccd9c-7fc2-4bdb-b409-aef8668e57ec None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.610548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcccd9c-7fc2-4bdb-b409-aef8668e57ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.622283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcccd9c-7fc2-4bdb-b409-aef8668e57ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:21.623448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d317a74-a412-4453-b994-4c0c084c7554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:21.624081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317a74-a412-4453-b994-4c0c084c7554 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.624380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317a74-a412-4453-b994-4c0c084c7554 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.624629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317a74-a412-4453-b994-4c0c084c7554 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-0fcccd9c-7fc2-4bdb-b409-aef8668e57ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:21.626900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:21.627763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:21.628573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 146/301] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:21.629279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcccd9c-7fc2-4bdb-b409-aef8668e57ec None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:21.642797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d317a74-a412-4453-b994-4c0c084c7554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-4d317a74-a412-4453-b994-4c0c084c7554 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:21.647916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:21.649468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:21.650452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 156/302] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:21.651179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d317a74-a412-4453-b994-4c0c084c7554 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:21.682460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:21.768452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcccd9c-7fc2-4bdb-b409-aef8668e57ec None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:21.773067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d317a74-a412-4453-b994-4c0c084c7554 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:21.812960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4692a95-bdc6-43fc-9684-54ba23ed9919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:21.813814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4692a95-bdc6-43fc-9684-54ba23ed9919 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.814123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4692a95-bdc6-43fc-9684-54ba23ed9919 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.814399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4692a95-bdc6-43fc-9684-54ba23ed9919 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.814547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:21.815361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.815639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.815795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4692a95-bdc6-43fc-9684-54ba23ed9919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:21.816203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.816636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:21.816948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:21.817208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4692a95-bdc6-43fc-9684-54ba23ed9919 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:21.818345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:21.822180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 147/303] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:21.822933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4692a95-bdc6-43fc-9684-54ba23ed9919 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:21.829795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 157/304] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:21.830673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:21.866043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:21.949810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4692a95-bdc6-43fc-9684-54ba23ed9919 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:21.959154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:21.960055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ea3a1-8865-4a32-a718-8a0bb0e5030f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:21.989532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:21.990191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.990474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.990746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.991467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:21.991772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:21.993902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.007603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 148/305] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:22.007603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.011171 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855afe33-4d22-493e-902e-07d10a0daab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.011806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855afe33-4d22-493e-902e-07d10a0daab5 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.012082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855afe33-4d22-493e-902e-07d10a0daab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.012348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855afe33-4d22-493e-902e-07d10a0daab5 None admin] PATH_INFO: `/v3/groups` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.012976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855afe33-4d22-493e-902e-07d10a0daab5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:22.014489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855afe33-4d22-493e-902e-07d10a0daab5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.125107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 158/306] 10.222.0.76 () {60 vars in 1134 bytes} [Wed Aug 24 17:29:21 2022] POST /identity/v3/groups => generated 229 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:22.125990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855afe33-4d22-493e-902e-07d10a0daab5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.207276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0d4a5-1de3-4e75-90e8-ed0725549d11 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.233253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:22.250153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed5e6c-9acf-4752-b63f-2189ff2325c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.250809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed5e6c-9acf-4752-b63f-2189ff2325c3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.251090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed5e6c-9acf-4752-b63f-2189ff2325c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.251363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed5e6c-9acf-4752-b63f-2189ff2325c3 None admin] PATH_INFO: `/v3/users/1a2506e814f746a1b3022cdd9ca7c29b` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.256530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855afe33-4d22-493e-902e-07d10a0daab5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.273172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed5e6c-9acf-4752-b63f-2189ff2325c3 None admin] RBAC: Authorizing `identity:get_user(user_id=1a2506e814f746a1b3022cdd9ca7c29b)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:22.275529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed5e6c-9acf-4752-b63f-2189ff2325c3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.277467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 149/307] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/users/1a2506e814f746a1b3022cdd9ca7c29b => generated 293 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:22.283693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed5e6c-9acf-4752-b63f-2189ff2325c3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.308036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e915b8b-25c0-41b7-a4b3-a58c105c35aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.311614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e915b8b-25c0-41b7-a4b3-a58c105c35aa None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.311614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e915b8b-25c0-41b7-a4b3-a58c105c35aa None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.311614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e915b8b-25c0-41b7-a4b3-a58c105c35aa None admin] PATH_INFO: `/v3/users/73e1d4589ca144449e971100b5d65917` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.312380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e915b8b-25c0-41b7-a4b3-a58c105c35aa None admin] RBAC: Authorizing `identity:get_user(user_id=73e1d4589ca144449e971100b5d65917)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:22.314549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e915b8b-25c0-41b7-a4b3-a58c105c35aa None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.322195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 159/308] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/users/73e1d4589ca144449e971100b5d65917 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:22.322825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:22.325883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e915b8b-25c0-41b7-a4b3-a58c105c35aa None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.411179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed5e6c-9acf-4752-b63f-2189ff2325c3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.453515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e915b8b-25c0-41b7-a4b3-a58c105c35aa None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.458687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f1625-101b-4cd3-9520-d15ba83a1476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.459408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f1625-101b-4cd3-9520-d15ba83a1476 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.459761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f1625-101b-4cd3-9520-d15ba83a1476 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.460107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f1625-101b-4cd3-9520-d15ba83a1476 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.460854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f1625-101b-4cd3-9520-d15ba83a1476 None admin] RBAC: Adding query filter params (role.id=f9e4fcd3595c43609a8c949b1996b02c, user.id=1a2506e814f746a1b3022cdd9ca7c29b, scope.system=all) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:22.461733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f1625-101b-4cd3-9520-d15ba83a1476 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:22.463438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f1625-101b-4cd3-9520-d15ba83a1476 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.469001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 150/309] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/role_assignments?user.id=1a2506e814f746a1b3022cdd9ca7c29b&scope.system=all&role.id=f9e4fcd3595c43609a8c949b1996b02c => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:22.470274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d9186-7862-48af-bd59-9bfab3ddb476 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.470560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d9186-7862-48af-bd59-9bfab3ddb476 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.470837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d9186-7862-48af-bd59-9bfab3ddb476 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.472389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 151/310] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:22.480005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc4af4-690d-4e72-a8e8-762882777ffc None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.480379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc4af4-690d-4e72-a8e8-762882777ffc None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.480723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc4af4-690d-4e72-a8e8-762882777ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.504858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-01fc4af4-690d-4e72-a8e8-762882777ffc None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:22.506398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-01fc4af4-690d-4e72-a8e8-762882777ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:22.508671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ec5ce-a463-4359-8958-55804ee79a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.509556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec5ce-a463-4359-8958-55804ee79a36 None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.509967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec5ce-a463-4359-8958-55804ee79a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.510316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec5ce-a463-4359-8958-55804ee79a36 None admin] PATH_INFO: `/v3/system/users/73e1d4589ca144449e971100b5d65917/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.512045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ec5ce-a463-4359-8958-55804ee79a36 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=73e1d4589ca144449e971100b5d65917, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:22.513485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ec5ce-a463-4359-8958-55804ee79a36 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.594543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 152/311] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:22.621919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 160/312] 10.222.0.76 () {58 vars in 1334 bytes} [Wed Aug 24 17:29:22 2022] PUT /identity/v3/system/users/73e1d4589ca144449e971100b5d65917/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:22.886907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc4af4-690d-4e72-a8e8-762882777ffc None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.916610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:23.031380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc4af4-690d-4e72-a8e8-762882777ffc None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:23.097431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aff5d-c142-4c94-aa30-9cb684c08f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:23.098097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aff5d-c142-4c94-aa30-9cb684c08f19 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:23.098380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aff5d-c142-4c94-aa30-9cb684c08f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:23.098649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aff5d-c142-4c94-aa30-9cb684c08f19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.116067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8aff5d-c142-4c94-aa30-9cb684c08f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-0d8aff5d-c142-4c94-aa30-9cb684c08f19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:23.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:23.122846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:23.123638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 153/313] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:23.134012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ec5ce-a463-4359-8958-55804ee79a36 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:23.158987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:23.264347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ec5ce-a463-4359-8958-55804ee79a36 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:23.304446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:23.306054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:23.306538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:23.307009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.307966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:23.308503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:23.310218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:23.326112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 161/314] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:23.331062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aff5d-c142-4c94-aa30-9cb684c08f19 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:23.445079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:23.466512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aff5d-c142-4c94-aa30-9cb684c08f19 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:23.514068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a6b0-31dd-4446-96be-f8f5c97d1715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:23.514716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a6b0-31dd-4446-96be-f8f5c97d1715 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:23.514995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a6b0-31dd-4446-96be-f8f5c97d1715 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:23.515268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a6b0-31dd-4446-96be-f8f5c97d1715 None admin] PATH_INFO: `/v3/users/02fd5af397ee4fd79406fd4017635d8e` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.516599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a6b0-31dd-4446-96be-f8f5c97d1715 None admin] RBAC: Authorizing `identity:get_user(user_id=02fd5af397ee4fd79406fd4017635d8e)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:23.518108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a6b0-31dd-4446-96be-f8f5c97d1715 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:23.520072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 154/315] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:23 2022] GET /identity/v3/users/02fd5af397ee4fd79406fd4017635d8e => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:23.526494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:23.632774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:23.655442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a11c70-d71d-4d6d-bfd8-c5001e9ad5ef None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:23.699970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e327-6568-478b-8f13-1052922ead7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:23.700523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e327-6568-478b-8f13-1052922ead7e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:23.700804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e327-6568-478b-8f13-1052922ead7e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:23.701082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e327-6568-478b-8f13-1052922ead7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.784219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2e327-6568-478b-8f13-1052922ead7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:23.786890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2e327-6568-478b-8f13-1052922ead7e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:23.894319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:23.921888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 162/316] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:23.925263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a6b0-31dd-4446-96be-f8f5c97d1715 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:24.072679 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a6b0-31dd-4446-96be-f8f5c97d1715 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:24.109643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217c1d-7943-45e4-b6c0-5c5afaaed351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:24.110314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217c1d-7943-45e4-b6c0-5c5afaaed351 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.110596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217c1d-7943-45e4-b6c0-5c5afaaed351 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.110925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217c1d-7943-45e4-b6c0-5c5afaaed351 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.112071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af217c1d-7943-45e4-b6c0-5c5afaaed351 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:24.113506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af217c1d-7943-45e4-b6c0-5c5afaaed351 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:24.115200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 155/317] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:23 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:24.121786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e327-6568-478b-8f13-1052922ead7e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:24.230284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e327-6568-478b-8f13-1052922ead7e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:24.268241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab9805-6f53-428a-adbe-b25dc71f721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:24.268852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab9805-6f53-428a-adbe-b25dc71f721a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.269133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab9805-6f53-428a-adbe-b25dc71f721a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.269405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab9805-6f53-428a-adbe-b25dc71f721a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.270179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab9805-6f53-428a-adbe-b25dc71f721a None admin] RBAC: Adding query filter params (role.id=f9e4fcd3595c43609a8c949b1996b02c, scope.project.id=a667e49b60584e6382b48fec8d858ac4, user.id=02fd5af397ee4fd79406fd4017635d8e) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:24.270490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab9805-6f53-428a-adbe-b25dc71f721a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:24.271954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab9805-6f53-428a-adbe-b25dc71f721a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:24.279122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 163/318] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:24 2022] GET /identity/v3/role_assignments?user.id=02fd5af397ee4fd79406fd4017635d8e&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=f9e4fcd3595c43609a8c949b1996b02c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:24.465236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114897c7-99b5-4ba7-983b-6cd9bdf7f05a None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.465749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114897c7-99b5-4ba7-983b-6cd9bdf7f05a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.466265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114897c7-99b5-4ba7-983b-6cd9bdf7f05a None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.469362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 156/319] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:24.478417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a66d5-15e0-413f-870e-e84808af9a16 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.478767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a66d5-15e0-413f-870e-e84808af9a16 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.478986 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a66d5-15e0-413f-870e-e84808af9a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.490625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b578a-4b10-4890-a718-a587f671ddb4 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.491324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b578a-4b10-4890-a718-a587f671ddb4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.491800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b578a-4b10-4890-a718-a587f671ddb4 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.494440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 157/320] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:24.502622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b82cc-693e-499f-ba96-f667786aac4d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.502622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b82cc-693e-499f-ba96-f667786aac4d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.502622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b82cc-693e-499f-ba96-f667786aac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.502941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-963a66d5-15e0-413f-870e-e84808af9a16 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:24.503179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-963a66d5-15e0-413f-870e-e84808af9a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:24.522044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-071b82cc-693e-499f-ba96-f667786aac4d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:24.523281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-071b82cc-693e-499f-ba96-f667786aac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:24.536208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:24.566512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:24.602712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 164/321] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:24.610034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 158/322] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:24.878083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a66d5-15e0-413f-870e-e84808af9a16 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:24.881820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842cced-f0fe-4f72-9f04-ade3d00c7465 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.882142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842cced-f0fe-4f72-9f04-ade3d00c7465 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.882420 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842cced-f0fe-4f72-9f04-ade3d00c7465 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.884021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 159/323] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:29:24.884908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842cced-f0fe-4f72-9f04-ade3d00c7465 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:24.997955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a66d5-15e0-413f-870e-e84808af9a16 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:24.998652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842cced-f0fe-4f72-9f04-ade3d00c7465 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.034381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168e58d-36ee-447e-84a9-dd08a587172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.034600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a02098-69e6-4dc4-a3d6-1fbb730d6a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.035020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e58d-36ee-447e-84a9-dd08a587172e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.035122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02098-69e6-4dc4-a3d6-1fbb730d6a0d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.035301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02098-69e6-4dc4-a3d6-1fbb730d6a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.035408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e58d-36ee-447e-84a9-dd08a587172e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.035572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e58d-36ee-447e-84a9-dd08a587172e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.035674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02098-69e6-4dc4-a3d6-1fbb730d6a0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.036692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168e58d-36ee-447e-84a9-dd08a587172e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.038213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168e58d-36ee-447e-84a9-dd08a587172e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:25.040459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 165/324] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:25.040459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36b9e8-6528-4350-94b3-1b0bb7b752c8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.042069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36b9e8-6528-4350-94b3-1b0bb7b752c8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.042278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36b9e8-6528-4350-94b3-1b0bb7b752c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.050602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a02098-69e6-4dc4-a3d6-1fbb730d6a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c4a02098-69e6-4dc4-a3d6-1fbb730d6a0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:25.055635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:25.056744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 160/325] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:25.058467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a02098-69e6-4dc4-a3d6-1fbb730d6a0d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.065097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-3b36b9e8-6528-4350-94b3-1b0bb7b752c8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:25.066475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3b36b9e8-6528-4350-94b3-1b0bb7b752c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:25.069333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:25.103979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:25.137286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 166/326] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:25.138408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36b9e8-6528-4350-94b3-1b0bb7b752c8 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.164458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a02098-69e6-4dc4-a3d6-1fbb730d6a0d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.202319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d430-4236-448a-9aa6-33ff48dba990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.202933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d430-4236-448a-9aa6-33ff48dba990 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.203257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d430-4236-448a-9aa6-33ff48dba990 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.203504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d430-4236-448a-9aa6-33ff48dba990 None admin] PATH_INFO: `/v3/groups` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.204110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168d430-4236-448a-9aa6-33ff48dba990 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.205540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168d430-4236-448a-9aa6-33ff48dba990 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:25.250224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36b9e8-6528-4350-94b3-1b0bb7b752c8 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.285223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.285838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.286155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.286427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.287101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:25.287402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.288843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:25.298618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 167/327] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:25.303996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.309742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 161/328] 10.222.0.76 () {60 vars in 1134 bytes} [Wed Aug 24 17:29:25 2022] POST /identity/v3/groups => generated 226 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:25.348540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:25.414590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d430-4236-448a-9aa6-33ff48dba990 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.434210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae417766-be19-4d0a-91cd-6b278304f943 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.475591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197396b-0d2e-4e13-a0b8-66bcdb14d81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.476237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197396b-0d2e-4e13-a0b8-66bcdb14d81f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.477139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197396b-0d2e-4e13-a0b8-66bcdb14d81f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.477359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197396b-0d2e-4e13-a0b8-66bcdb14d81f None admin] PATH_INFO: `/v3/users/1a2506e814f746a1b3022cdd9ca7c29b` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.479712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197396b-0d2e-4e13-a0b8-66bcdb14d81f None admin] RBAC: Authorizing `identity:get_user(user_id=1a2506e814f746a1b3022cdd9ca7c29b)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.481781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197396b-0d2e-4e13-a0b8-66bcdb14d81f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:25.483602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 168/329] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/users/1a2506e814f746a1b3022cdd9ca7c29b => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:25.489347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197396b-0d2e-4e13-a0b8-66bcdb14d81f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.500830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:25.543727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d430-4236-448a-9aa6-33ff48dba990 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.588666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2fa7d-ebf7-4aed-ac3f-faa995dd3881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.589323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2fa7d-ebf7-4aed-ac3f-faa995dd3881 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.589607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2fa7d-ebf7-4aed-ac3f-faa995dd3881 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.589998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2fa7d-ebf7-4aed-ac3f-faa995dd3881 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.607564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2fa7d-ebf7-4aed-ac3f-faa995dd3881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-34b2fa7d-ebf7-4aed-ac3f-faa995dd3881 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:25.613342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:25.614352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:25.615203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 162/330] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:25.621616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2fa7d-ebf7-4aed-ac3f-faa995dd3881 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.640124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197396b-0d2e-4e13-a0b8-66bcdb14d81f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.641824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:25.687331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b2b11-4ed2-4572-bc7d-884321a411d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.688015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2b11-4ed2-4572-bc7d-884321a411d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.688365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2b11-4ed2-4572-bc7d-884321a411d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.688719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2b11-4ed2-4572-bc7d-884321a411d0 None admin] PATH_INFO: `/v3/system/users/1a2506e814f746a1b3022cdd9ca7c29b/roles/f9e4fcd3595c43609a8c949b1996b02c` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.691212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b2b11-4ed2-4572-bc7d-884321a411d0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1a2506e814f746a1b3022cdd9ca7c29b, role_id=f9e4fcd3595c43609a8c949b1996b02c)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.694930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b2b11-4ed2-4572-bc7d-884321a411d0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:25.760298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 169/331] 10.222.0.76 () {58 vars in 1334 bytes} [Wed Aug 24 17:29:25 2022] PUT /identity/v3/system/users/1a2506e814f746a1b3022cdd9ca7c29b/roles/f9e4fcd3595c43609a8c949b1996b02c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:29:25.762019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2fa7d-ebf7-4aed-ac3f-faa995dd3881 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.796208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:25.808793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.809427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.809714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.810033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.810765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:25.811077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.812516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:25.825838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 163/332] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:25.831795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b2b11-4ed2-4572-bc7d-884321a411d0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.973121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b2b11-4ed2-4572-bc7d-884321a411d0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:26.012585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70d06-8ae3-48b3-bc10-196a3517291f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:26.013451 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70d06-8ae3-48b3-bc10-196a3517291f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:26.013789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70d06-8ae3-48b3-bc10-196a3517291f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:26.014099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70d06-8ae3-48b3-bc10-196a3517291f None admin] PATH_INFO: `/v3/users/73e1d4589ca144449e971100b5d65917` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:26.015538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b70d06-8ae3-48b3-bc10-196a3517291f None admin] RBAC: Authorizing `identity:get_user(user_id=73e1d4589ca144449e971100b5d65917)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:26.017042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b70d06-8ae3-48b3-bc10-196a3517291f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:26.022935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 170/333] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/users/73e1d4589ca144449e971100b5d65917 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:26.026112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:26.138165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b93e-23eb-4ccb-9ec7-561e7e1d83fd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:26.181519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085981e-8954-4b1c-b25a-c79ccb1141d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:26.182173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085981e-8954-4b1c-b25a-c79ccb1141d3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:26.182446 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085981e-8954-4b1c-b25a-c79ccb1141d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:26.182711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085981e-8954-4b1c-b25a-c79ccb1141d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:26.183463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085981e-8954-4b1c-b25a-c79ccb1141d3 None admin] RBAC: Adding query filter params (role.id=fff136f214f2428895daf10483c542d1, user.id=73e1d4589ca144449e971100b5d65917, scope.system=all) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:26.183767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085981e-8954-4b1c-b25a-c79ccb1141d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:26.185220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085981e-8954-4b1c-b25a-c79ccb1141d3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:26.202050 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 164/334] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 24 17:29:26 2022] GET /identity/v3/role_assignments?user.id=73e1d4589ca144449e971100b5d65917&scope.system=all&role.id=fff136f214f2428895daf10483c542d1 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:27.557164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6ef2d-a11b-4f64-a55a-0d2f4582c4f8 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.557684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6ef2d-a11b-4f64-a55a-0d2f4582c4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.557760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6ef2d-a11b-4f64-a55a-0d2f4582c4f8 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:27.559142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 171/335] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:27.565183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e454068-bc71-4f6f-ba85-f57b89a21c2f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.565679 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e454068-bc71-4f6f-ba85-f57b89a21c2f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.565784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e454068-bc71-4f6f-ba85-f57b89a21c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:27.592447 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-8e454068-bc71-4f6f-ba85-f57b89a21c2f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:27.593660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8e454068-bc71-4f6f-ba85-f57b89a21c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:27.648926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:27.683266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 165/336] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:27.963676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30fcf-f12f-45e8-ae0f-374887f4d49b None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.963959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30fcf-f12f-45e8-ae0f-374887f4d49b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.964236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30fcf-f12f-45e8-ae0f-374887f4d49b None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:27.965973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 172/337] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:27.980324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b555f9-a45c-4d70-ba39-b66d776dabf6 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.980903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b555f9-a45c-4d70-ba39-b66d776dabf6 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.981356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b555f9-a45c-4d70-ba39-b66d776dabf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.006350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f2b555f9-a45c-4d70-ba39-b66d776dabf6 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:28.007867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f2b555f9-a45c-4d70-ba39-b66d776dabf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:28.092472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 166/338] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:28.184303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f30fcf-f12f-45e8-ae0f-374887f4d49b None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.228492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.306566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f30fcf-f12f-45e8-ae0f-374887f4d49b None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.356898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d9249-bd6a-49e5-b1d0-ea4b59a4f259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.357912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d9249-bd6a-49e5-b1d0-ea4b59a4f259 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.358406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d9249-bd6a-49e5-b1d0-ea4b59a4f259 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.358847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d9249-bd6a-49e5-b1d0-ea4b59a4f259 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.371706 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d9249-bd6a-49e5-b1d0-ea4b59a4f259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-1b1d9249-bd6a-49e5-b1d0-ea4b59a4f259 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:28.377313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:28.378251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:28.379044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:28.385576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 173/339] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:28.386965 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b555f9-a45c-4d70-ba39-b66d776dabf6 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.436312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d9249-bd6a-49e5-b1d0-ea4b59a4f259 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.623270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b555f9-a45c-4d70-ba39-b66d776dabf6 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.634545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.635067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.645093 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d9249-bd6a-49e5-b1d0-ea4b59a4f259 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.695862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9509bd0-3d97-4e70-922d-3409f7db264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.696436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9509bd0-3d97-4e70-922d-3409f7db264c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.696728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9509bd0-3d97-4e70-922d-3409f7db264c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.697001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9509bd0-3d97-4e70-922d-3409f7db264c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.699273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05efbc50-1ee9-4dd5-b450-04d87e9d4889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.699886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efbc50-1ee9-4dd5-b450-04d87e9d4889 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.700164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efbc50-1ee9-4dd5-b450-04d87e9d4889 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.700435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efbc50-1ee9-4dd5-b450-04d87e9d4889 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.701121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05efbc50-1ee9-4dd5-b450-04d87e9d4889 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:28.701426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05efbc50-1ee9-4dd5-b450-04d87e9d4889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.703246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05efbc50-1ee9-4dd5-b450-04d87e9d4889 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.718594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 167/340] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:28.720033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce605-a9e7-47c6-9b96-d65e6729b72e None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.720321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce605-a9e7-47c6-9b96-d65e6729b72e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.720594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce605-a9e7-47c6-9b96-d65e6729b72e None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.721930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 168/341] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:28.722648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ce605-a9e7-47c6-9b96-d65e6729b72e None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.724769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9509bd0-3d97-4e70-922d-3409f7db264c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-e9509bd0-3d97-4e70-922d-3409f7db264c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:28.731874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:28.732770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:28.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:28.740135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 174/342] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:28.747409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848529e-1e36-4f02-9b2c-8b44f02d8a3c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.747959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848529e-1e36-4f02-9b2c-8b44f02d8a3c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.748447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848529e-1e36-4f02-9b2c-8b44f02d8a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.770523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e848529e-1e36-4f02-9b2c-8b44f02d8a3c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:28.772264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e848529e-1e36-4f02-9b2c-8b44f02d8a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:28.850418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 175/343] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:28.852580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848529e-1e36-4f02-9b2c-8b44f02d8a3c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.858601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ce605-a9e7-47c6-9b96-d65e6729b72e None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.942478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f15d6-9c0f-40a8-8c35-fe018ad172bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.943105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f15d6-9c0f-40a8-8c35-fe018ad172bb None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.943385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f15d6-9c0f-40a8-8c35-fe018ad172bb None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.943657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f15d6-9c0f-40a8-8c35-fe018ad172bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.016611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.020752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.051105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848529e-1e36-4f02-9b2c-8b44f02d8a3c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.087368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f15d6-9c0f-40a8-8c35-fe018ad172bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.088972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f15d6-9c0f-40a8-8c35-fe018ad172bb None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.106835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6679a2a-bdeb-454f-8217-1de9d8434746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.107461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6679a2a-bdeb-454f-8217-1de9d8434746 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.107742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6679a2a-bdeb-454f-8217-1de9d8434746 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.108015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6679a2a-bdeb-454f-8217-1de9d8434746 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.108701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6679a2a-bdeb-454f-8217-1de9d8434746 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:29.109003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6679a2a-bdeb-454f-8217-1de9d8434746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.110499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6679a2a-bdeb-454f-8217-1de9d8434746 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.126655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 176/344] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:29.128025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827a970-02a0-45dc-afba-ccd04d809ecc None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.128217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827a970-02a0-45dc-afba-ccd04d809ecc None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.128497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827a970-02a0-45dc-afba-ccd04d809ecc None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.135683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 177/345] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:29.136507 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827a970-02a0-45dc-afba-ccd04d809ecc None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.276989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 169/346] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:29.278317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400f28-9ba3-4eab-976f-6f160033b19c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.278607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400f28-9ba3-4eab-976f-6f160033b19c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.282207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400f28-9ba3-4eab-976f-6f160033b19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.283662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827a970-02a0-45dc-afba-ccd04d809ecc None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.307237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-78400f28-9ba3-4eab-976f-6f160033b19c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:29.308659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-78400f28-9ba3-4eab-976f-6f160033b19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:29.329556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fd00c-4add-413f-ab61-854204fecbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.330447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd00c-4add-413f-ab61-854204fecbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.330804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd00c-4add-413f-ab61-854204fecbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.331240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd00c-4add-413f-ab61-854204fecbb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.351383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.384544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.393152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 170/347] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:29.394071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400f28-9ba3-4eab-976f-6f160033b19c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.425658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fd00c-4add-413f-ab61-854204fecbb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.427338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fd00c-4add-413f-ab61-854204fecbb5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.544902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400f28-9ba3-4eab-976f-6f160033b19c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.610824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 178/348] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:29.611972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fd00c-4add-413f-ab61-854204fecbb5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.651185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9318fe6-5822-4d62-90df-f1bafa902e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.651865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9318fe6-5822-4d62-90df-f1bafa902e17 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.652147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9318fe6-5822-4d62-90df-f1bafa902e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.652417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9318fe6-5822-4d62-90df-f1bafa902e17 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.675365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9318fe6-5822-4d62-90df-f1bafa902e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e9318fe6-5822-4d62-90df-f1bafa902e17 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:29.688346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:29.689332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:29:29.690427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:29.691711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 171/349] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:29.693440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9318fe6-5822-4d62-90df-f1bafa902e17 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.699290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.778909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.801881 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fd00c-4add-413f-ab61-854204fecbb5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.838443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9318fe6-5822-4d62-90df-f1bafa902e17 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.853788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85359a95-8b31-46dc-aabb-155a62a68cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.854573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85359a95-8b31-46dc-aabb-155a62a68cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.854860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85359a95-8b31-46dc-aabb-155a62a68cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.855132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85359a95-8b31-46dc-aabb-155a62a68cb2 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.856493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85359a95-8b31-46dc-aabb-155a62a68cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.858306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85359a95-8b31-46dc-aabb-155a62a68cb2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.860156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 179/350] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:29.860928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85359a95-8b31-46dc-aabb-155a62a68cb2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.886672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e080778-73f6-4484-be7b-1c4036429281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.887333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e080778-73f6-4484-be7b-1c4036429281 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.887621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e080778-73f6-4484-be7b-1c4036429281 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.887899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e080778-73f6-4484-be7b-1c4036429281 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.889171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e080778-73f6-4484-be7b-1c4036429281 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.891497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e080778-73f6-4484-be7b-1c4036429281 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.892868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 172/351] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:29.893561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e080778-73f6-4484-be7b-1c4036429281 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.015485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85359a95-8b31-46dc-aabb-155a62a68cb2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.033195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:30.045206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e080778-73f6-4484-be7b-1c4036429281 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.060141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.060982 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.061418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.061691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.062470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:30.065213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.065213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:30.078174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 180/352] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:30.079541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.083987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:30.099387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccbd09-fb1e-4706-ae07-777cedf570fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.100884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccbd09-fb1e-4706-ae07-777cedf570fd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.101887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccbd09-fb1e-4706-ae07-777cedf570fd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.103548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccbd09-fb1e-4706-ae07-777cedf570fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.117574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccbd09-fb1e-4706-ae07-777cedf570fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-71ccbd09-fb1e-4706-ae07-777cedf570fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:30.122664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:30.123640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:30.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 173/353] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:30.125185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccbd09-fb1e-4706-ae07-777cedf570fd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.234181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef60d90-8c93-439d-a7b9-565f64024bca None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.261458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccbd09-fb1e-4706-ae07-777cedf570fd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.284151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889785b5-670a-40c2-be4b-f2be8f4063d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.284810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889785b5-670a-40c2-be4b-f2be8f4063d5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.285099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889785b5-670a-40c2-be4b-f2be8f4063d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.285373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889785b5-670a-40c2-be4b-f2be8f4063d5 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.294823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889785b5-670a-40c2-be4b-f2be8f4063d5 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.296307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889785b5-670a-40c2-be4b-f2be8f4063d5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:30.298232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 181/354] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:30.299279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b16818-3f15-4eea-be6a-7e3cc8296086 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.299562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b16818-3f15-4eea-be6a-7e3cc8296086 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.299839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b16818-3f15-4eea-be6a-7e3cc8296086 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.301451 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 182/355] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:30.301827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b16818-3f15-4eea-be6a-7e3cc8296086 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.315555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab0ecb-37a4-4558-a6d3-ffd791875184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.316234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0ecb-37a4-4558-a6d3-ffd791875184 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.316522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0ecb-37a4-4558-a6d3-ffd791875184 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.317180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0ecb-37a4-4558-a6d3-ffd791875184 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.318669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab0ecb-37a4-4558-a6d3-ffd791875184 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.320267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab0ecb-37a4-4558-a6d3-ffd791875184 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:30.322760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 174/356] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:30.324039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44060286-3e22-48cd-bfca-9de7d7baf6ab None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.324415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44060286-3e22-48cd-bfca-9de7d7baf6ab None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.324687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44060286-3e22-48cd-bfca-9de7d7baf6ab None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.326170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 175/357] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:30.326790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44060286-3e22-48cd-bfca-9de7d7baf6ab None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.361327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:30.406364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:30.443572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b16818-3f15-4eea-be6a-7e3cc8296086 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.464098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44060286-3e22-48cd-bfca-9de7d7baf6ab None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.491642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdbf0f-8aa8-42ef-a319-092f4ab3eee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.493384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdbf0f-8aa8-42ef-a319-092f4ab3eee9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.493667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdbf0f-8aa8-42ef-a319-092f4ab3eee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.494135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdbf0f-8aa8-42ef-a319-092f4ab3eee9 None admin] PATH_INFO: `/v3/users/1a2506e814f746a1b3022cdd9ca7c29b` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.495994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdbf0f-8aa8-42ef-a319-092f4ab3eee9 None admin] RBAC: Authorizing `identity:get_user(user_id=1a2506e814f746a1b3022cdd9ca7c29b)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.497321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdbf0f-8aa8-42ef-a319-092f4ab3eee9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:30.499773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 183/358] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/users/1a2506e814f746a1b3022cdd9ca7c29b => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:30.501224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdbf0f-8aa8-42ef-a319-092f4ab3eee9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.515393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685aed4-ff0a-4571-b4c7-58134041cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.515865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685aed4-ff0a-4571-b4c7-58134041cb09 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.516581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685aed4-ff0a-4571-b4c7-58134041cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.516581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685aed4-ff0a-4571-b4c7-58134041cb09 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.517084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685aed4-ff0a-4571-b4c7-58134041cb09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:30.517420 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685aed4-ff0a-4571-b4c7-58134041cb09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.518818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685aed4-ff0a-4571-b4c7-58134041cb09 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:30.530873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 176/359] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:30.534989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec21ff-1aeb-4f10-9fc0-72e22a310606 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.534989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec21ff-1aeb-4f10-9fc0-72e22a310606 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.534989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec21ff-1aeb-4f10-9fc0-72e22a310606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.552894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-68ec21ff-1aeb-4f10-9fc0-72e22a310606 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:30.554814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-68ec21ff-1aeb-4f10-9fc0-72e22a310606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:30.642879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdbf0f-8aa8-42ef-a319-092f4ab3eee9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.654541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:30.657294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 177/360] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:30.658583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec21ff-1aeb-4f10-9fc0-72e22a310606 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.704083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81ebfc-9f45-4e79-9bdf-8a13c9c468f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.705419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81ebfc-9f45-4e79-9bdf-8a13c9c468f3 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.705705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81ebfc-9f45-4e79-9bdf-8a13c9c468f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.706513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81ebfc-9f45-4e79-9bdf-8a13c9c468f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.707134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81ebfc-9f45-4e79-9bdf-8a13c9c468f3 None admin] RBAC: Adding query filter params (group.id=5867ea6696b441ac9ea74d0733b6393d, role.id=fff136f214f2428895daf10483c542d1, scope.project.id=2f9e413c56d445ab804ef9a518cc2202) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:30.707379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81ebfc-9f45-4e79-9bdf-8a13c9c468f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.708870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81ebfc-9f45-4e79-9bdf-8a13c9c468f3 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:30.721000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 184/361] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/role_assignments?group.id=5867ea6696b441ac9ea74d0733b6393d&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=fff136f214f2428895daf10483c542d1 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:30.722799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8e560-cfeb-4352-869b-e534932db239 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.722799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8e560-cfeb-4352-869b-e534932db239 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.723090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8e560-cfeb-4352-869b-e534932db239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.735381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:30.749126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-fff8e560-cfeb-4352-869b-e534932db239 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:30.753356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-fff8e560-cfeb-4352-869b-e534932db239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:30.811459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec21ff-1aeb-4f10-9fc0-72e22a310606 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.844141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 185/362] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:30.845668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8e560-cfeb-4352-869b-e534932db239 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.877717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.878782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.879264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.879822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.880926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] RBAC: Adding query filter params (group.id=5867ea6696b441ac9ea74d0733b6393d, role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=2f9e413c56d445ab804ef9a518cc2202) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:30.881451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.883215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:30.897523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 178/363] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/role_assignments?group.id=5867ea6696b441ac9ea74d0733b6393d&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=d6fe1aca91e74aad9907c855501f588f => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:30.899448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.996656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8e560-cfeb-4352-869b-e534932db239 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.998418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.044744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.045435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.045723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.046048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.046793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] RBAC: Adding query filter params (role.id=f9e4fcd3595c43609a8c949b1996b02c, user.id=1a2506e814f746a1b3022cdd9ca7c29b, scope.system=all) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:31.047101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:31.053037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:31.066448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 186/364] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/role_assignments?user.id=1a2506e814f746a1b3022cdd9ca7c29b&scope.system=all&role.id=f9e4fcd3595c43609a8c949b1996b02c => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:31.067101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.105357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5caa-92fe-45ee-8048-7f9db6b43cc0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.121320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.151130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79915a-c7fd-48c2-96e6-821c2ae925b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.152167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79915a-c7fd-48c2-96e6-821c2ae925b0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.152684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79915a-c7fd-48c2-96e6-821c2ae925b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.153306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79915a-c7fd-48c2-96e6-821c2ae925b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.227102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e4559-d082-4aac-ba8c-a2b5fc21c745 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.259750 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79915a-c7fd-48c2-96e6-821c2ae925b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:31.261466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79915a-c7fd-48c2-96e6-821c2ae925b0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:31.278780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc0017-a2cc-404b-851e-082191186666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.279835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc0017-a2cc-404b-851e-082191186666 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.280342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc0017-a2cc-404b-851e-082191186666 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.280861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc0017-a2cc-404b-851e-082191186666 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.297021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc0017-a2cc-404b-851e-082191186666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-7afc0017-a2cc-404b-851e-082191186666 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:31.303411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:31.304434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:31.305538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:31.308059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 187/365] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:31.310019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc0017-a2cc-404b-851e-082191186666 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.339339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.401660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 179/366] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:31.402916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79915a-c7fd-48c2-96e6-821c2ae925b0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.419656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.454308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc0017-a2cc-404b-851e-082191186666 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.508361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddba662-6d1d-4593-b53b-38df9da73839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.509176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddba662-6d1d-4593-b53b-38df9da73839 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.509536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddba662-6d1d-4593-b53b-38df9da73839 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.509816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddba662-6d1d-4593-b53b-38df9da73839 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.530477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79915a-c7fd-48c2-96e6-821c2ae925b0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.531592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddba662-6d1d-4593-b53b-38df9da73839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-3ddba662-6d1d-4593-b53b-38df9da73839 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:31.537185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:31.538183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 188/367] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:31.539951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddba662-6d1d-4593-b53b-38df9da73839 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.573824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.575058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.575936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.576558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.577537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:31.578228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:31.580455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:31.592342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 180/368] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:31.594147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.640996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.664912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddba662-6d1d-4593-b53b-38df9da73839 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.676972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.707855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd7c72-eb08-40be-9b70-86ec4cae0dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.709079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd7c72-eb08-40be-9b70-86ec4cae0dfd None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.709528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd7c72-eb08-40be-9b70-86ec4cae0dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.710002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd7c72-eb08-40be-9b70-86ec4cae0dfd None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.712181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd7c72-eb08-40be-9b70-86ec4cae0dfd None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:31.713846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd7c72-eb08-40be-9b70-86ec4cae0dfd None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:31.714095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108db896-f847-4a9e-ab55-fefaabf78062 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.716882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 189/369] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:31.718033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd7c72-eb08-40be-9b70-86ec4cae0dfd None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.753389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.754142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.754342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.754605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.755320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:31.755628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:31.757068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:31.768523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 181/370] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:31.769319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.837970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd7c72-eb08-40be-9b70-86ec4cae0dfd None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.878620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327dfce-3e14-4518-8c2a-3d15b3013e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.879648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327dfce-3e14-4518-8c2a-3d15b3013e26 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.880149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327dfce-3e14-4518-8c2a-3d15b3013e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.880659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327dfce-3e14-4518-8c2a-3d15b3013e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.880938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881fae-d25b-4ede-8fc0-a89628451e6a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.902038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.921771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ba66-15d7-4389-969d-df49b8428a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.922487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ba66-15d7-4389-969d-df49b8428a12 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.922772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ba66-15d7-4389-969d-df49b8428a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.923071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ba66-15d7-4389-969d-df49b8428a12 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.924213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284ba66-15d7-4389-969d-df49b8428a12 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:31.925661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284ba66-15d7-4389-969d-df49b8428a12 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:31.927323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 182/371] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:31.928209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ba66-15d7-4389-969d-df49b8428a12 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.931886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.971424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327dfce-3e14-4518-8c2a-3d15b3013e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:31.973274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327dfce-3e14-4518-8c2a-3d15b3013e26 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.042650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ba66-15d7-4389-969d-df49b8428a12 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.079417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618458-a4c6-4fb0-813c-68974c8dd656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.080033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618458-a4c6-4fb0-813c-68974c8dd656 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.080320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618458-a4c6-4fb0-813c-68974c8dd656 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.080662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618458-a4c6-4fb0-813c-68974c8dd656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.089499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 190/372] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:32.090820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327dfce-3e14-4518-8c2a-3d15b3013e26 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.146751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.158650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07618458-a4c6-4fb0-813c-68974c8dd656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.160174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07618458-a4c6-4fb0-813c-68974c8dd656 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.162315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.208118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327dfce-3e14-4518-8c2a-3d15b3013e26 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.247571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.248191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.248530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.248810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.249704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] RBAC: Adding query filter params (group.id=5867ea6696b441ac9ea74d0733b6393d, role.id=fff136f214f2428895daf10483c542d1, scope.project.id=a667e49b60584e6382b48fec8d858ac4) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:32.250048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.251624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.259118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 191/373] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/role_assignments?group.id=5867ea6696b441ac9ea74d0733b6393d&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=fff136f214f2428895daf10483c542d1 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:32.259884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.278110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 183/374] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:32.283930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618458-a4c6-4fb0-813c-68974c8dd656 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.385152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0e988-948f-49c2-a1e1-befaea7060f2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.400595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.404601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618458-a4c6-4fb0-813c-68974c8dd656 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.406733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.426234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0557d1-b703-4a93-9844-5fe6edeadd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.426889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0557d1-b703-4a93-9844-5fe6edeadd37 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.427171 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0557d1-b703-4a93-9844-5fe6edeadd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.427447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0557d1-b703-4a93-9844-5fe6edeadd37 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.428662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0557d1-b703-4a93-9844-5fe6edeadd37 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.431185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0557d1-b703-4a93-9844-5fe6edeadd37 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.433129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 192/375] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:32.440014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0557d1-b703-4a93-9844-5fe6edeadd37 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.471007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5871f4-d72b-419f-a687-bb7fc4081ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.471322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5871f4-d72b-419f-a687-bb7fc4081ffc None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.472502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5871f4-d72b-419f-a687-bb7fc4081ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.472946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5871f4-d72b-419f-a687-bb7fc4081ffc None admin] PATH_INFO: `/v3/projects/74aebe409e204672bb76c767e1d15f99` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.475644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5871f4-d72b-419f-a687-bb7fc4081ffc None admin] RBAC: Authorizing `identity:get_project(project_id=74aebe409e204672bb76c767e1d15f99)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.478014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5871f4-d72b-419f-a687-bb7fc4081ffc None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.484025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 184/376] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/projects/74aebe409e204672bb76c767e1d15f99 => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:32.489673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5871f4-d72b-419f-a687-bb7fc4081ffc None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.612937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0557d1-b703-4a93-9844-5fe6edeadd37 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.644736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5871f4-d72b-419f-a687-bb7fc4081ffc None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.660803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f69a2-34db-4df4-90ea-e5877f09ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.662084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f69a2-34db-4df4-90ea-e5877f09ce61 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.662601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f69a2-34db-4df4-90ea-e5877f09ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.663086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f69a2-34db-4df4-90ea-e5877f09ce61 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.665457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f69a2-34db-4df4-90ea-e5877f09ce61 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.667770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f69a2-34db-4df4-90ea-e5877f09ce61 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.670544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 193/377] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:32.677932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f69a2-34db-4df4-90ea-e5877f09ce61 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.690798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e06e7-7596-46a8-85d5-b748ef8128ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.691500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e06e7-7596-46a8-85d5-b748ef8128ae None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.691791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e06e7-7596-46a8-85d5-b748ef8128ae None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.692070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e06e7-7596-46a8-85d5-b748ef8128ae None admin] PATH_INFO: `/v3/groups/28c61d4c046b4723b25b87a383f65245` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.700608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e06e7-7596-46a8-85d5-b748ef8128ae None admin] RBAC: Authorizing `identity:get_group(group_id=28c61d4c046b4723b25b87a383f65245)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.702086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e06e7-7596-46a8-85d5-b748ef8128ae None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.704452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 185/378] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/groups/28c61d4c046b4723b25b87a383f65245 => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:32.711487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e06e7-7596-46a8-85d5-b748ef8128ae None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.747946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.754265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.820414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f69a2-34db-4df4-90ea-e5877f09ce61 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.848945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e06e7-7596-46a8-85d5-b748ef8128ae None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.874363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c59ee-9fef-44fc-9946-86b12915d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.875402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c59ee-9fef-44fc-9946-86b12915d6dd None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.875882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c59ee-9fef-44fc-9946-86b12915d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.876366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c59ee-9fef-44fc-9946-86b12915d6dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.877313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c59ee-9fef-44fc-9946-86b12915d6dd None admin] RBAC: Adding query filter params (group.id=5867ea6696b441ac9ea74d0733b6393d, role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=a667e49b60584e6382b48fec8d858ac4) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:32.877824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c59ee-9fef-44fc-9946-86b12915d6dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.879583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c59ee-9fef-44fc-9946-86b12915d6dd None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.888724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 194/379] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/role_assignments?group.id=5867ea6696b441ac9ea74d0733b6393d&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=d6fe1aca91e74aad9907c855501f588f => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:32.896086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374b4d3-a4c0-44b6-8dc2-61077a8cc90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.896728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b4d3-a4c0-44b6-8dc2-61077a8cc90f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.897015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b4d3-a4c0-44b6-8dc2-61077a8cc90f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.897294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b4d3-a4c0-44b6-8dc2-61077a8cc90f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.898163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374b4d3-a4c0-44b6-8dc2-61077a8cc90f None admin] RBAC: Adding query filter params (group.id=28c61d4c046b4723b25b87a383f65245, role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=74aebe409e204672bb76c767e1d15f99) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:32.898483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374b4d3-a4c0-44b6-8dc2-61077a8cc90f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.899998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374b4d3-a4c0-44b6-8dc2-61077a8cc90f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.907170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 186/380] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/role_assignments?group.id=28c61d4c046b4723b25b87a383f65245&scope.project.id=74aebe409e204672bb76c767e1d15f99&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:33.409689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741aabb-ae36-4441-9d14-28223f7e1c54 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.410092 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741aabb-ae36-4441-9d14-28223f7e1c54 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.410421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741aabb-ae36-4441-9d14-28223f7e1c54 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.412092 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 195/381] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:33.418629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1170d-cc84-40b3-b633-57ec0e9458ac None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.418911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1170d-cc84-40b3-b633-57ec0e9458ac None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.419215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1170d-cc84-40b3-b633-57ec0e9458ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.441365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-2bb1170d-cc84-40b3-b633-57ec0e9458ac None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:33.442705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2bb1170d-cc84-40b3-b633-57ec0e9458ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:33.538260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 187/382] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:33.663203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc6fe-86e1-4396-84a7-6522206e79cb None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.663514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc6fe-86e1-4396-84a7-6522206e79cb None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.664220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc6fe-86e1-4396-84a7-6522206e79cb None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.666005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 196/383] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:33.671287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e4fd-5ed1-46b2-95a5-7d29e2a5858a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.671628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e4fd-5ed1-46b2-95a5-7d29e2a5858a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.671888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e4fd-5ed1-46b2-95a5-7d29e2a5858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.698996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-8c96e4fd-5ed1-46b2-95a5-7d29e2a5858a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:33.700980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8c96e4fd-5ed1-46b2-95a5-7d29e2a5858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:33.727874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:33.784617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 188/384] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:33.875348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facc6fe-86e1-4396-84a7-6522206e79cb None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.027338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facc6fe-86e1-4396-84a7-6522206e79cb None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.050978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:34.074929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd76ba-d4c1-4cee-8438-d1f7067dc9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.075569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd76ba-d4c1-4cee-8438-d1f7067dc9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.075847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd76ba-d4c1-4cee-8438-d1f7067dc9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.076176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd76ba-d4c1-4cee-8438-d1f7067dc9e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.090107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd76ba-d4c1-4cee-8438-d1f7067dc9e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-bafd76ba-d4c1-4cee-8438-d1f7067dc9e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:34.095267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:34.096183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:34.096960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:34.099399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 197/385] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:34.105964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96e4fd-5ed1-46b2-95a5-7d29e2a5858a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.221175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96e4fd-5ed1-46b2-95a5-7d29e2a5858a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.237550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd76ba-d4c1-4cee-8438-d1f7067dc9e4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.295904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.296523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.296805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.297081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.297759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:34.298088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.301303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.323164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 189/386] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:34.326302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.372237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:34.412191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd76ba-d4c1-4cee-8438-d1f7067dc9e4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.466423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c1f0f-c79c-4184-bd51-bcaf9ebb65b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.467078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c1f0f-c79c-4184-bd51-bcaf9ebb65b2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.467358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c1f0f-c79c-4184-bd51-bcaf9ebb65b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.467629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c1f0f-c79c-4184-bd51-bcaf9ebb65b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.485539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c1f0f-c79c-4184-bd51-bcaf9ebb65b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-b91c1f0f-c79c-4184-bd51-bcaf9ebb65b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:34.490981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:34.493836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:34.495853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:34.497349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d724e76-f002-440a-a177-fa459e50f9ef None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.497496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 198/387] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:34.503768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c1f0f-c79c-4184-bd51-bcaf9ebb65b2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.548497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dbb10-e524-4e65-a5dc-badbf5098c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.549149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dbb10-e524-4e65-a5dc-badbf5098c15 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.549531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dbb10-e524-4e65-a5dc-badbf5098c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.549814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dbb10-e524-4e65-a5dc-badbf5098c15 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.551023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dbb10-e524-4e65-a5dc-badbf5098c15 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.552602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dbb10-e524-4e65-a5dc-badbf5098c15 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.560316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:34.561022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 190/388] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:34 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:34.562313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dbb10-e524-4e65-a5dc-badbf5098c15 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.712438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:34.733285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c1f0f-c79c-4184-bd51-bcaf9ebb65b2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.738205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dbb10-e524-4e65-a5dc-badbf5098c15 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.807543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.808160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.808443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.808717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.809895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1677-9845-4f27-891a-b943543e4b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.810580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1677-9845-4f27-891a-b943543e4b77 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.810862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1677-9845-4f27-891a-b943543e4b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.811131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1677-9845-4f27-891a-b943543e4b77 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.811889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:34.812202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.813596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.815349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd1677-9845-4f27-891a-b943543e4b77 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.816878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd1677-9845-4f27-891a-b943543e4b77 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.818604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 191/389] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:34 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:34.825153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1677-9845-4f27-891a-b943543e4b77 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.832040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 199/390] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:34.838597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.006905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:35.015355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bad265-8566-41fb-94a1-a019fe29e658 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.045735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1677-9845-4f27-891a-b943543e4b77 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.056213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd935ad-d8ca-4363-b410-aacc7d66baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.056856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd935ad-d8ca-4363-b410-aacc7d66baf7 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.057142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd935ad-d8ca-4363-b410-aacc7d66baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.057420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd935ad-d8ca-4363-b410-aacc7d66baf7 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.061882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd935ad-d8ca-4363-b410-aacc7d66baf7 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.061882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd935ad-d8ca-4363-b410-aacc7d66baf7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.062246 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 200/391] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:34 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:35.068562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd935ad-d8ca-4363-b410-aacc7d66baf7 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.110358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf164cf-991f-49e9-84c1-9ac908a86e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.111516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf164cf-991f-49e9-84c1-9ac908a86e37 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.111810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf164cf-991f-49e9-84c1-9ac908a86e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.112236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf164cf-991f-49e9-84c1-9ac908a86e37 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202/groups/5867ea6696b441ac9ea74d0733b6393d/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.115011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf164cf-991f-49e9-84c1-9ac908a86e37 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f9e413c56d445ab804ef9a518cc2202, group_id=5867ea6696b441ac9ea74d0733b6393d, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.116906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf164cf-991f-49e9-84c1-9ac908a86e37 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.184569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 192/392] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 17:29:34 2022] PUT /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202/groups/5867ea6696b441ac9ea74d0733b6393d/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:35.186011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793beea2-541d-41a5-9081-5e13e4e39f83 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.186303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793beea2-541d-41a5-9081-5e13e4e39f83 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.186594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793beea2-541d-41a5-9081-5e13e4e39f83 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.188196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 193/393] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:35.199097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed740cc1-9b2f-4e79-9f43-5bd5cb4eeeaa None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.199401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed740cc1-9b2f-4e79-9f43-5bd5cb4eeeaa None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.199745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed740cc1-9b2f-4e79-9f43-5bd5cb4eeeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.210003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd935ad-d8ca-4363-b410-aacc7d66baf7 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.237348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ed740cc1-9b2f-4e79-9f43-5bd5cb4eeeaa None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:35.238631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ed740cc1-9b2f-4e79-9f43-5bd5cb4eeeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:35.252214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359f181-b28a-4691-99e9-6e5c32f04c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.252893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359f181-b28a-4691-99e9-6e5c32f04c38 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.253175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359f181-b28a-4691-99e9-6e5c32f04c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.253452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359f181-b28a-4691-99e9-6e5c32f04c38 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.254713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359f181-b28a-4691-99e9-6e5c32f04c38 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.256176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359f181-b28a-4691-99e9-6e5c32f04c38 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.258534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 201/394] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:35 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:35.258916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:35.264326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359f181-b28a-4691-99e9-6e5c32f04c38 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.306385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:35.374362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 194/395] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:35.421343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359f181-b28a-4691-99e9-6e5c32f04c38 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.458183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d425d98-900d-484a-b87b-78610a85cb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.459233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d425d98-900d-484a-b87b-78610a85cb0e None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.459650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d425d98-900d-484a-b87b-78610a85cb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.460091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d425d98-900d-484a-b87b-78610a85cb0e None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202/groups/5867ea6696b441ac9ea74d0733b6393d/roles/d6fe1aca91e74aad9907c855501f588f` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.463218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d425d98-900d-484a-b87b-78610a85cb0e None admin] RBAC: Authorizing `identity:create_grant(project_id=2f9e413c56d445ab804ef9a518cc2202, group_id=5867ea6696b441ac9ea74d0733b6393d, role_id=d6fe1aca91e74aad9907c855501f588f)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.464907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d425d98-900d-484a-b87b-78610a85cb0e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.521448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dec14-f0d4-4def-a8e4-9deb35612754 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.522006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dec14-f0d4-4def-a8e4-9deb35612754 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.522550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dec14-f0d4-4def-a8e4-9deb35612754 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.525055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 195/396] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:29:35.529894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf138-996e-4325-b2bb-11fa64faf6c0 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.530190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf138-996e-4325-b2bb-11fa64faf6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.530520 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf138-996e-4325-b2bb-11fa64faf6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.554430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-846bf138-996e-4325-b2bb-11fa64faf6c0 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:35.554616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 202/397] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 17:29:35 2022] PUT /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202/groups/5867ea6696b441ac9ea74d0733b6393d/roles/d6fe1aca91e74aad9907c855501f588f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:29:35.555775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-846bf138-996e-4325-b2bb-11fa64faf6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:35.665122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 196/398] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:35.670983 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d425d98-900d-484a-b87b-78610a85cb0e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.762299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a41ab-5e00-4ec6-90c5-a465d02f9514 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.762807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a41ab-5e00-4ec6-90c5-a465d02f9514 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.762897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a41ab-5e00-4ec6-90c5-a465d02f9514 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.764187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 197/399] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:29:35.771023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1cdbb-d278-4498-a161-d4ffc6c24952 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.771516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1cdbb-d278-4498-a161-d4ffc6c24952 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.771827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1cdbb-d278-4498-a161-d4ffc6c24952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.793417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:35.796894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ede1cdbb-d278-4498-a161-d4ffc6c24952 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:35.798288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ede1cdbb-d278-4498-a161-d4ffc6c24952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:35.805143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d425d98-900d-484a-b87b-78610a85cb0e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.855134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a38e4-6023-4b94-a6a9-628da8f2f1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.855922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a38e4-6023-4b94-a6a9-628da8f2f1ae None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.856204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a38e4-6023-4b94-a6a9-628da8f2f1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.856474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a38e4-6023-4b94-a6a9-628da8f2f1ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.879770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a38e4-6023-4b94-a6a9-628da8f2f1ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:35.884655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 198/400] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-1e5a38e4-6023-4b94-a6a9-628da8f2f1ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:35.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:35.887602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:35.889508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 203/401] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:35.899137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1cdbb-d278-4498-a161-d4ffc6c24952 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.910226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:36.031950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a38e4-6023-4b94-a6a9-628da8f2f1ae None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.068018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1cdbb-d278-4498-a161-d4ffc6c24952 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.150275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a38e4-6023-4b94-a6a9-628da8f2f1ae None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.167313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.168089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.168509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.168790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.169719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:36.170053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.171841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.185939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 199/402] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:36.192841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.201361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ce35a-4da8-498f-ae35-ec069afe8d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.202025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ce35a-4da8-498f-ae35-ec069afe8d31 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.202311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ce35a-4da8-498f-ae35-ec069afe8d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.202678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ce35a-4da8-498f-ae35-ec069afe8d31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.224457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ce35a-4da8-498f-ae35-ec069afe8d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-ff9ce35a-4da8-498f-ae35-ec069afe8d31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:36.230670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:36.231544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:36.232341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:36.232985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 204/403] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:36.241218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ce35a-4da8-498f-ae35-ec069afe8d31 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.258312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:36.298456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:36.333741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a783aa-5aa8-432c-aa9f-8dd35319ecc6 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.371446 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18cf2-f33a-4dfd-bb96-26371962ee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.372198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18cf2-f33a-4dfd-bb96-26371962ee55 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.372543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18cf2-f33a-4dfd-bb96-26371962ee55 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.372885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18cf2-f33a-4dfd-bb96-26371962ee55 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.374079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb18cf2-f33a-4dfd-bb96-26371962ee55 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.375559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb18cf2-f33a-4dfd-bb96-26371962ee55 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.377496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 200/404] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:36.383432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18cf2-f33a-4dfd-bb96-26371962ee55 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.387348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ce35a-4da8-498f-ae35-ec069afe8d31 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.423827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.424557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.424842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.425115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.425928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:36.426236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.427772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.443352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 205/405] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:36.443352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.521515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18cf2-f33a-4dfd-bb96-26371962ee55 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.524827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:36.566265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0913bbe-1502-43ab-a330-94828162ff22 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.567776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39e312-f126-40b1-84ca-5855cd048d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.568763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39e312-f126-40b1-84ca-5855cd048d4c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.570587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39e312-f126-40b1-84ca-5855cd048d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.571066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39e312-f126-40b1-84ca-5855cd048d4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.575030 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:36.584577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39e312-f126-40b1-84ca-5855cd048d4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-af39e312-f126-40b1-84ca-5855cd048d4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:36.590441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:36.595154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 201/406] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:36.595993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39e312-f126-40b1-84ca-5855cd048d4c None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.609107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b11ee1-eac0-4763-bf7a-ea8ad5dc4672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.609797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11ee1-eac0-4763-bf7a-ea8ad5dc4672 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.610112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11ee1-eac0-4763-bf7a-ea8ad5dc4672 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.610395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11ee1-eac0-4763-bf7a-ea8ad5dc4672 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.612788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b11ee1-eac0-4763-bf7a-ea8ad5dc4672 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.614686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b11ee1-eac0-4763-bf7a-ea8ad5dc4672 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.617381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 206/407] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:36.622434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b11ee1-eac0-4763-bf7a-ea8ad5dc4672 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.716249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39e312-f126-40b1-84ca-5855cd048d4c None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.753502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441198c-ee63-4a80-86b0-4dd0cc62ffd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.754588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441198c-ee63-4a80-86b0-4dd0cc62ffd6 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.755282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441198c-ee63-4a80-86b0-4dd0cc62ffd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.755752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441198c-ee63-4a80-86b0-4dd0cc62ffd6 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.757727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441198c-ee63-4a80-86b0-4dd0cc62ffd6 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.759666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441198c-ee63-4a80-86b0-4dd0cc62ffd6 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.766423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b11ee1-eac0-4763-bf7a-ea8ad5dc4672 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.766797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 202/408] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:36.768172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441198c-ee63-4a80-86b0-4dd0cc62ffd6 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.809962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.810591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.810874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.811146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.811819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:36.812125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.813496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.826443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 207/409] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:36.827468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.830950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:36.849122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:36.901647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441198c-ee63-4a80-86b0-4dd0cc62ffd6 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.943618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0e68f-0dac-409e-b85b-e16f0eebfcff None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.959667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbb0ee-036f-422b-b497-1c6fbc4e401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.960359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbb0ee-036f-422b-b497-1c6fbc4e401d None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.961275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbb0ee-036f-422b-b497-1c6fbc4e401d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.961275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbb0ee-036f-422b-b497-1c6fbc4e401d None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4/groups/5867ea6696b441ac9ea74d0733b6393d/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.963623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbb0ee-036f-422b-b497-1c6fbc4e401d None admin] RBAC: Authorizing `identity:create_grant(project_id=a667e49b60584e6382b48fec8d858ac4, group_id=5867ea6696b441ac9ea74d0733b6393d, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.965252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbb0ee-036f-422b-b497-1c6fbc4e401d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.984222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63759ab-2885-4fc5-9f7e-14667515dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.984936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63759ab-2885-4fc5-9f7e-14667515dac2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.985389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63759ab-2885-4fc5-9f7e-14667515dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.985772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63759ab-2885-4fc5-9f7e-14667515dac2 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.986984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63759ab-2885-4fc5-9f7e-14667515dac2 None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.988479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63759ab-2885-4fc5-9f7e-14667515dac2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.990441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 208/410] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:36.996319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63759ab-2885-4fc5-9f7e-14667515dac2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:37.075818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:37.090743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 203/411] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 17:29:36 2022] PUT /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4/groups/5867ea6696b441ac9ea74d0733b6393d/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:37.092058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbb0ee-036f-422b-b497-1c6fbc4e401d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:37.140477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63759ab-2885-4fc5-9f7e-14667515dac2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:37.221263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13119109-f450-4d34-8af3-e2e11ac37997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:37.221938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13119109-f450-4d34-8af3-e2e11ac37997 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.222223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13119109-f450-4d34-8af3-e2e11ac37997 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.222499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13119109-f450-4d34-8af3-e2e11ac37997 None admin] PATH_INFO: `/v3/groups/28c61d4c046b4723b25b87a383f65245` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.227235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13119109-f450-4d34-8af3-e2e11ac37997 None admin] RBAC: Authorizing `identity:get_group(group_id=28c61d4c046b4723b25b87a383f65245)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:37.228762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13119109-f450-4d34-8af3-e2e11ac37997 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:37.230532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 209/412] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/groups/28c61d4c046b4723b25b87a383f65245 => generated 229 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:37.237921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13119109-f450-4d34-8af3-e2e11ac37997 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:37.295682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbb0ee-036f-422b-b497-1c6fbc4e401d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:37.322269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:37.333960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61514424-787a-4238-bab9-3d3ed07e5449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:37.334678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61514424-787a-4238-bab9-3d3ed07e5449 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.336248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61514424-787a-4238-bab9-3d3ed07e5449 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.336248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61514424-787a-4238-bab9-3d3ed07e5449 None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4/groups/5867ea6696b441ac9ea74d0733b6393d/roles/d6fe1aca91e74aad9907c855501f588f` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.337758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61514424-787a-4238-bab9-3d3ed07e5449 None admin] RBAC: Authorizing `identity:create_grant(project_id=a667e49b60584e6382b48fec8d858ac4, group_id=5867ea6696b441ac9ea74d0733b6393d, role_id=d6fe1aca91e74aad9907c855501f588f)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:37.339497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61514424-787a-4238-bab9-3d3ed07e5449 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:37.371390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13119109-f450-4d34-8af3-e2e11ac37997 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:37.419214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 204/413] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 17:29:37 2022] PUT /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4/groups/5867ea6696b441ac9ea74d0733b6393d/roles/d6fe1aca91e74aad9907c855501f588f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:37.420409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af6c4c-34d5-4368-a96f-8fe9ab00a777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:37.421150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af6c4c-34d5-4368-a96f-8fe9ab00a777 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.421429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af6c4c-34d5-4368-a96f-8fe9ab00a777 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.421700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af6c4c-34d5-4368-a96f-8fe9ab00a777 None admin] PATH_INFO: `/v3/projects/74aebe409e204672bb76c767e1d15f99` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.426333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af6c4c-34d5-4368-a96f-8fe9ab00a777 None admin] RBAC: Authorizing `identity:get_project(project_id=74aebe409e204672bb76c767e1d15f99)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:37.426499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7298-542c-4bd7-a4b2-34dd444dcd2d None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.426699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7298-542c-4bd7-a4b2-34dd444dcd2d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.426892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7298-542c-4bd7-a4b2-34dd444dcd2d None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.427804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af6c4c-34d5-4368-a96f-8fe9ab00a777 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:37.428646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 205/414] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:37.435233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 210/415] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:37 2022] GET /identity/v3/projects/74aebe409e204672bb76c767e1d15f99 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:37.437338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c36da-7597-44ea-8d4b-5511afac5c76 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.437628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c36da-7597-44ea-8d4b-5511afac5c76 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.437924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c36da-7597-44ea-8d4b-5511afac5c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.441483 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af6c4c-34d5-4368-a96f-8fe9ab00a777 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:37.459579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:37.479186 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d70c36da-7597-44ea-8d4b-5511afac5c76 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:37.480442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d70c36da-7597-44ea-8d4b-5511afac5c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:37.601568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 206/416] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:37.640068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af6c4c-34d5-4368-a96f-8fe9ab00a777 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:37.677547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac8706-ad5c-4940-b49b-da880448435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:37.678343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac8706-ad5c-4940-b49b-da880448435c None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.678629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac8706-ad5c-4940-b49b-da880448435c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.678908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac8706-ad5c-4940-b49b-da880448435c None admin] PATH_INFO: `/v3/projects/74aebe409e204672bb76c767e1d15f99/groups/28c61d4c046b4723b25b87a383f65245/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.681487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac8706-ad5c-4940-b49b-da880448435c None admin] RBAC: Authorizing `identity:create_grant(project_id=74aebe409e204672bb76c767e1d15f99, group_id=28c61d4c046b4723b25b87a383f65245, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:37.682985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac8706-ad5c-4940-b49b-da880448435c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:37.749072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 211/417] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 17:29:37 2022] PUT /identity/v3/projects/74aebe409e204672bb76c767e1d15f99/groups/28c61d4c046b4723b25b87a383f65245/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:37.782032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb33fc-28be-4ac3-bace-cb1574c94f4f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.782977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb33fc-28be-4ac3-bace-cb1574c94f4f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.783472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb33fc-28be-4ac3-bace-cb1574c94f4f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.785023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 207/418] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:37.791631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a1a02-7c04-470e-8b7a-78c2c7e63262 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.792221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a1a02-7c04-470e-8b7a-78c2c7e63262 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.792661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a1a02-7c04-470e-8b7a-78c2c7e63262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.812993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-326a1a02-7c04-470e-8b7a-78c2c7e63262 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:37.815044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-326a1a02-7c04-470e-8b7a-78c2c7e63262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:37.889670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:37.891154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb33fc-28be-4ac3-bace-cb1574c94f4f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:37.902028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 212/419] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:38.158621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb33fc-28be-4ac3-bace-cb1574c94f4f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:38.176725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:38.196199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a1a02-7c04-470e-8b7a-78c2c7e63262 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:38.210318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e35d82-f100-49d7-88d8-e24d59fde10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:38.210966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e35d82-f100-49d7-88d8-e24d59fde10c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:38.211244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e35d82-f100-49d7-88d8-e24d59fde10c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:38.211513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e35d82-f100-49d7-88d8-e24d59fde10c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:38.233895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e35d82-f100-49d7-88d8-e24d59fde10c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-a3e35d82-f100-49d7-88d8-e24d59fde10c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:38.238697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:38.239584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:38.240375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:38.241583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 208/420] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:38.248293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e35d82-f100-49d7-88d8-e24d59fde10c None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:38.382911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e35d82-f100-49d7-88d8-e24d59fde10c None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:38.398664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a1a02-7c04-470e-8b7a-78c2c7e63262 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:38.441959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:38.441959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:38.441959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:38.441959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:38.442345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] RBAC: Adding query filter params (name=member) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:38.442345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:38.442345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:38.451398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822f7e1-1e93-4233-a82c-948bc7fadc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:38.451854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 209/421] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:38.452141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f7e1-1e93-4233-a82c-948bc7fadc48 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:38.452428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f7e1-1e93-4233-a82c-948bc7fadc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:38.452702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f7e1-1e93-4233-a82c-948bc7fadc48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:38.461145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:38.481192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822f7e1-1e93-4233-a82c-948bc7fadc48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-0822f7e1-1e93-4233-a82c-948bc7fadc48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:38.486241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:38.487112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:38.487885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:38.489751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 213/422] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:38.496399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822f7e1-1e93-4233-a82c-948bc7fadc48 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:38.550219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:38.604613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:38.648539 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5bc87-46c8-4a51-80f1-fc4d2d2a1761 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:38.685585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822f7e1-1e93-4233-a82c-948bc7fadc48 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:38.715286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ca61-bce2-45e4-b2bf-785fb5924059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:38.715911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ca61-bce2-45e4-b2bf-785fb5924059 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:38.716192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ca61-bce2-45e4-b2bf-785fb5924059 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:38.716465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ca61-bce2-45e4-b2bf-785fb5924059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:38.751657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:38.752271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:38.752547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:38.752816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:38.753492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:38.753789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:38.755271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:38.769882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 214/423] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:38.775520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:38.847362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964ca61-bce2-45e4-b2bf-785fb5924059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:38.848939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964ca61-bce2-45e4-b2bf-785fb5924059 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:38.920988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:38.968148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93e184-71f9-4fba-a225-3f8865141d94 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.006223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:39.038249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a26b10-e463-4593-9f6e-de579ce046e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.038897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a26b10-e463-4593-9f6e-de579ce046e8 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.039176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a26b10-e463-4593-9f6e-de579ce046e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.039449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a26b10-e463-4593-9f6e-de579ce046e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.046334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 210/424] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:29:39.048994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ca61-bce2-45e4-b2bf-785fb5924059 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:39.167328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a26b10-e463-4593-9f6e-de579ce046e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:39.168800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a26b10-e463-4593-9f6e-de579ce046e8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:39.185093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ca61-bce2-45e4-b2bf-785fb5924059 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.264264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b370d63-afa9-4b08-8f8b-850921060b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.264827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b370d63-afa9-4b08-8f8b-850921060b5d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.265109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b370d63-afa9-4b08-8f8b-850921060b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.265383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b370d63-afa9-4b08-8f8b-850921060b5d None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.268745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b370d63-afa9-4b08-8f8b-850921060b5d None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:39.270221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b370d63-afa9-4b08-8f8b-850921060b5d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:39.275313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 211/425] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:39 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:39.279115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b370d63-afa9-4b08-8f8b-850921060b5d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:39.286385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:39.358316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 215/426] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:29:39.360137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a26b10-e463-4593-9f6e-de579ce046e8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:39.382381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:39.466532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b370d63-afa9-4b08-8f8b-850921060b5d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.472716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a26b10-e463-4593-9f6e-de579ce046e8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.502437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdebf6c-a49a-45f9-917d-f3d38ee2e136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.503309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdebf6c-a49a-45f9-917d-f3d38ee2e136 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.503734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdebf6c-a49a-45f9-917d-f3d38ee2e136 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.504147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdebf6c-a49a-45f9-917d-f3d38ee2e136 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.505400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdebf6c-a49a-45f9-917d-f3d38ee2e136 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:39.507793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdebf6c-a49a-45f9-917d-f3d38ee2e136 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:39.510515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 212/427] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:39 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:39.511819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5af46-2590-4c0f-8dd6-97ca30c4c939 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.512245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5af46-2590-4c0f-8dd6-97ca30c4c939 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.512661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5af46-2590-4c0f-8dd6-97ca30c4c939 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.514045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 213/428] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:39.514375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586da0c-d57e-44e8-b461-5cfdd0bdd917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.515192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586da0c-d57e-44e8-b461-5cfdd0bdd917 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.515611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586da0c-d57e-44e8-b461-5cfdd0bdd917 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.516021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586da0c-d57e-44e8-b461-5cfdd0bdd917 None admin] PATH_INFO: `/v3/projects/2f9e413c56d445ab804ef9a518cc2202` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.516841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5af46-2590-4c0f-8dd6-97ca30c4c939 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:39.517486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586da0c-d57e-44e8-b461-5cfdd0bdd917 None admin] RBAC: Authorizing `identity:get_project(project_id=2f9e413c56d445ab804ef9a518cc2202)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:39.519025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586da0c-d57e-44e8-b461-5cfdd0bdd917 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:39.521181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 216/429] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:39 2022] GET /identity/v3/projects/2f9e413c56d445ab804ef9a518cc2202 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:39.527594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa87408-bb45-40d1-bfd5-4f4d3b2dbbd0 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.528088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa87408-bb45-40d1-bfd5-4f4d3b2dbbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.528503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa87408-bb45-40d1-bfd5-4f4d3b2dbbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.546905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7fa87408-bb45-40d1-bfd5-4f4d3b2dbbd0 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:39.548318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7fa87408-bb45-40d1-bfd5-4f4d3b2dbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:39.609596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:39.640597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 217/430] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:39.641935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa87408-bb45-40d1-bfd5-4f4d3b2dbbd0 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:39.642773 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5af46-2590-4c0f-8dd6-97ca30c4c939 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.667071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:39.681080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bfb47-6b4a-45e3-b81c-b2bf8fc39e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.682018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bfb47-6b4a-45e3-b81c-b2bf8fc39e92 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.682464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bfb47-6b4a-45e3-b81c-b2bf8fc39e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.682926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bfb47-6b4a-45e3-b81c-b2bf8fc39e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.683834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bfb47-6b4a-45e3-b81c-b2bf8fc39e92 None admin] RBAC: Adding query filter params (group.id=5867ea6696b441ac9ea74d0733b6393d, role.id=fff136f214f2428895daf10483c542d1, scope.project.id=2f9e413c56d445ab804ef9a518cc2202) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:39.684369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bfb47-6b4a-45e3-b81c-b2bf8fc39e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:39.686042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bfb47-6b4a-45e3-b81c-b2bf8fc39e92 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:39.698448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 214/431] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:39 2022] GET /identity/v3/role_assignments?group.id=5867ea6696b441ac9ea74d0733b6393d&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=fff136f214f2428895daf10483c542d1 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:39.735237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1f853-bc52-498a-82a8-1c05a0823b33 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.735528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1f853-bc52-498a-82a8-1c05a0823b33 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.735807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1f853-bc52-498a-82a8-1c05a0823b33 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.737170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 215/432] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:39.743468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876cc6-b672-4621-86ff-af91c66d3073 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.743763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876cc6-b672-4621-86ff-af91c66d3073 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.743987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876cc6-b672-4621-86ff-af91c66d3073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.769645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa87408-bb45-40d1-bfd5-4f4d3b2dbbd0 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.781615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c8876cc6-b672-4621-86ff-af91c66d3073 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:39.782984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c8876cc6-b672-4621-86ff-af91c66d3073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:39.810729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdef6b-37b7-47c9-89ed-eaa7b2210f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.811760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdef6b-37b7-47c9-89ed-eaa7b2210f2b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.812279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdef6b-37b7-47c9-89ed-eaa7b2210f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.812842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdef6b-37b7-47c9-89ed-eaa7b2210f2b None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.814709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdef6b-37b7-47c9-89ed-eaa7b2210f2b None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:39.816495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdef6b-37b7-47c9-89ed-eaa7b2210f2b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:39.818844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 218/433] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:39 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:39.825953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdef6b-37b7-47c9-89ed-eaa7b2210f2b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:39.931396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:39.940553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 216/434] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:39.955021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdef6b-37b7-47c9-89ed-eaa7b2210f2b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.997203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b5d45-2cec-464d-873d-54cc6f37bdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.997824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b5d45-2cec-464d-873d-54cc6f37bdee None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.998138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b5d45-2cec-464d-873d-54cc6f37bdee None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.998412 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b5d45-2cec-464d-873d-54cc6f37bdee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.999153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b5d45-2cec-464d-873d-54cc6f37bdee None admin] RBAC: Adding query filter params (group.id=5867ea6696b441ac9ea74d0733b6393d, role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=2f9e413c56d445ab804ef9a518cc2202) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:39.999462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b5d45-2cec-464d-873d-54cc6f37bdee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:40.001056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b5d45-2cec-464d-873d-54cc6f37bdee None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:40.008784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 219/435] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:39 2022] GET /identity/v3/role_assignments?group.id=5867ea6696b441ac9ea74d0733b6393d&scope.project.id=2f9e413c56d445ab804ef9a518cc2202&role.id=d6fe1aca91e74aad9907c855501f588f => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:40.024843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8876cc6-b672-4621-86ff-af91c66d3073 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:40.112390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:40.160612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8876cc6-b672-4621-86ff-af91c66d3073 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:40.201647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08387f82-77fd-436a-a7eb-dba2f24be695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:40.202417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387f82-77fd-436a-a7eb-dba2f24be695 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.202692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387f82-77fd-436a-a7eb-dba2f24be695 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.202970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387f82-77fd-436a-a7eb-dba2f24be695 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:40.217585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08387f82-77fd-436a-a7eb-dba2f24be695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:40.220130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0567-59fb-433f-a23a-9013f7b2df22 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.220421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0567-59fb-433f-a23a-9013f7b2df22 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.220700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0567-59fb-433f-a23a-9013f7b2df22 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:40.222243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 220/436] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-08387f82-77fd-436a-a7eb-dba2f24be695 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:40.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:40.223298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:40.224066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 217/437] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:40.229925 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c0567-59fb-433f-a23a-9013f7b2df22 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:40.234280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a6c88-b0b3-4729-bb16-dd8598e846fd None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.234562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a6c88-b0b3-4729-bb16-dd8598e846fd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.234835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a6c88-b0b3-4729-bb16-dd8598e846fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:40.255577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-974a6c88-b0b3-4729-bb16-dd8598e846fd None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:40.256754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-974a6c88-b0b3-4729-bb16-dd8598e846fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:40.339087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 218/438] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:40.339716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a6c88-b0b3-4729-bb16-dd8598e846fd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:40.361526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c0567-59fb-433f-a23a-9013f7b2df22 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:40.405372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:40.406023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.406368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.406702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:40.407383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] RBAC: Adding query filter params (name=member) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:40.407693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:40.409094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:40.416350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:40.423249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 221/439] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:29:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:40.424786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:40.439773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:40.482939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a6c88-b0b3-4729-bb16-dd8598e846fd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:40.532517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35323e9-9188-4654-a1dc-5ca0e44f7992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:40.533516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35323e9-9188-4654-a1dc-5ca0e44f7992 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.534130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35323e9-9188-4654-a1dc-5ca0e44f7992 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.534629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35323e9-9188-4654-a1dc-5ca0e44f7992 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:40.549890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35323e9-9188-4654-a1dc-5ca0e44f7992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-b35323e9-9188-4654-a1dc-5ca0e44f7992 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:40.556293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:40.557280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:40.558620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 219/440] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:40.566322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35323e9-9188-4654-a1dc-5ca0e44f7992 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:40.570902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77570197-c15b-4ad7-b009-6992d4bef539 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:40.619040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8699e-50e8-4a85-9ce8-0dcf79ffbb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:40.619583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8699e-50e8-4a85-9ce8-0dcf79ffbb65 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.619866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8699e-50e8-4a85-9ce8-0dcf79ffbb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.620147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8699e-50e8-4a85-9ce8-0dcf79ffbb65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:40.690727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35323e9-9188-4654-a1dc-5ca0e44f7992 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:40.701250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8699e-50e8-4a85-9ce8-0dcf79ffbb65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:40.702828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8699e-50e8-4a85-9ce8-0dcf79ffbb65 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:40.708146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:40.714820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:40.725895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:40.726965 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.727245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.727684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:40.728266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:40.728569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:40.729997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:40.738264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 220/441] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:40.739048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:40.806735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 222/442] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:29:40.807533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8699e-50e8-4a85-9ce8-0dcf79ffbb65 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:40.843878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bab4c-bb6c-47cc-8379-93124b2e7cc7 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:40.879328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ab461-3a95-4d08-ad73-0e82515d97b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:40.879882 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ab461-3a95-4d08-ad73-0e82515d97b5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.880167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ab461-3a95-4d08-ad73-0e82515d97b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.880444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ab461-3a95-4d08-ad73-0e82515d97b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:40.891764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ab461-3a95-4d08-ad73-0e82515d97b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-9a4ab461-3a95-4d08-ad73-0e82515d97b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:40.896534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:40.897522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:40.899131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 221/443] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:40.899840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ab461-3a95-4d08-ad73-0e82515d97b5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:40.910506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8699e-50e8-4a85-9ce8-0dcf79ffbb65 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:40.923416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:40.941339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:40.947713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b8d3-b78e-4e72-8dc5-69b141ca7c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:40.948326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b8d3-b78e-4e72-8dc5-69b141ca7c95 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:40.948611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b8d3-b78e-4e72-8dc5-69b141ca7c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:40.948886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b8d3-b78e-4e72-8dc5-69b141ca7c95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.002113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ab461-3a95-4d08-ad73-0e82515d97b5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.021508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5b8d3-b78e-4e72-8dc5-69b141ca7c95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.023853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5b8d3-b78e-4e72-8dc5-69b141ca7c95 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.037388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ed7e2-4064-4518-83ef-310a5188005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.038051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ed7e2-4064-4518-83ef-310a5188005b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.038438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ed7e2-4064-4518-83ef-310a5188005b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.038717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ed7e2-4064-4518-83ef-310a5188005b None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.039857 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ed7e2-4064-4518-83ef-310a5188005b None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.041359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ed7e2-4064-4518-83ef-310a5188005b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.043054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 222/444] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:40 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:41.043813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ed7e2-4064-4518-83ef-310a5188005b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.122182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 223/445] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:41.122849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b8d3-b78e-4e72-8dc5-69b141ca7c95 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.135416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:41.148902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ed7e2-4064-4518-83ef-310a5188005b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.156399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:41.184342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.184947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.185226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.185500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.186200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:41.186503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.187867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.196638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 223/446] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:41.197429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.227233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b8d3-b78e-4e72-8dc5-69b141ca7c95 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.261570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946af48-67ee-4345-8c3f-7114ed5caf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.262233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946af48-67ee-4345-8c3f-7114ed5caf80 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.262515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946af48-67ee-4345-8c3f-7114ed5caf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.262795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946af48-67ee-4345-8c3f-7114ed5caf80 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.263891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946af48-67ee-4345-8c3f-7114ed5caf80 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.265321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946af48-67ee-4345-8c3f-7114ed5caf80 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.266917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 224/447] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:41.267683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946af48-67ee-4345-8c3f-7114ed5caf80 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.298475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be45619-b25a-4f73-81db-1007ebbffcbc None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.332958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ada2b6-2902-4e74-abd2-71f3dd79941e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.333587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada2b6-2902-4e74-abd2-71f3dd79941e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.333887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada2b6-2902-4e74-abd2-71f3dd79941e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.334165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada2b6-2902-4e74-abd2-71f3dd79941e None admin] PATH_INFO: `/v3/projects/a667e49b60584e6382b48fec8d858ac4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.335834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ada2b6-2902-4e74-abd2-71f3dd79941e None admin] RBAC: Authorizing `identity:get_project(project_id=a667e49b60584e6382b48fec8d858ac4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.337259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ada2b6-2902-4e74-abd2-71f3dd79941e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.338956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 224/448] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/projects/a667e49b60584e6382b48fec8d858ac4 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:41.339802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ada2b6-2902-4e74-abd2-71f3dd79941e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.348574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:41.370181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946af48-67ee-4345-8c3f-7114ed5caf80 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.374890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:41.406721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddcb52-3b86-4f59-b8ae-bb3072f8c267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.406721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddcb52-3b86-4f59-b8ae-bb3072f8c267 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.406721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddcb52-3b86-4f59-b8ae-bb3072f8c267 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.406721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddcb52-3b86-4f59-b8ae-bb3072f8c267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.438407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ada2b6-2902-4e74-abd2-71f3dd79941e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.474263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddcb52-3b86-4f59-b8ae-bb3072f8c267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.475408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.475744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddcb52-3b86-4f59-b8ae-bb3072f8c267 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.476278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.476659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.476942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.477687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] RBAC: Adding query filter params (group.id=5867ea6696b441ac9ea74d0733b6393d, role.id=fff136f214f2428895daf10483c542d1, scope.project.id=a667e49b60584e6382b48fec8d858ac4) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:41.478031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.479459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.486024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 225/449] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/role_assignments?group.id=5867ea6696b441ac9ea74d0733b6393d&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=fff136f214f2428895daf10483c542d1 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:41.487190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.567503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:41.596247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 225/450] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:41.602436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddcb52-3b86-4f59-b8ae-bb3072f8c267 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.621733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:41.625365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc761096-ddca-45c7-bcdd-6c52e70d95a1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.668677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7677020-79e7-4c40-8a75-7305fedb4d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.669244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677020-79e7-4c40-8a75-7305fedb4d01 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.669516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677020-79e7-4c40-8a75-7305fedb4d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.669798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677020-79e7-4c40-8a75-7305fedb4d01 None admin] PATH_INFO: `/v3/groups/5867ea6696b441ac9ea74d0733b6393d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.671020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7677020-79e7-4c40-8a75-7305fedb4d01 None admin] RBAC: Authorizing `identity:get_group(group_id=5867ea6696b441ac9ea74d0733b6393d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.673228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7677020-79e7-4c40-8a75-7305fedb4d01 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.674778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 226/451] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/groups/5867ea6696b441ac9ea74d0733b6393d => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:41.682433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7677020-79e7-4c40-8a75-7305fedb4d01 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.723494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddcb52-3b86-4f59-b8ae-bb3072f8c267 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.765336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0da38-2c73-4687-a6a5-820ec83a8d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.766067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0da38-2c73-4687-a6a5-820ec83a8d95 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.766353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0da38-2c73-4687-a6a5-820ec83a8d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.766632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0da38-2c73-4687-a6a5-820ec83a8d95 None admin] PATH_INFO: `/v3/projects/74aebe409e204672bb76c767e1d15f99` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.767799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0da38-2c73-4687-a6a5-820ec83a8d95 None admin] RBAC: Authorizing `identity:get_project(project_id=74aebe409e204672bb76c767e1d15f99)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.769274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0da38-2c73-4687-a6a5-820ec83a8d95 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.771131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 226/452] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/projects/74aebe409e204672bb76c767e1d15f99 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:41.777315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0da38-2c73-4687-a6a5-820ec83a8d95 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.814951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7677020-79e7-4c40-8a75-7305fedb4d01 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.833981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:41.854208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.854817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.855106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.855381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.856125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] RBAC: Adding query filter params (group.id=5867ea6696b441ac9ea74d0733b6393d, role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=a667e49b60584e6382b48fec8d858ac4) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:41.856437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.857905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.863833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 227/453] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/role_assignments?group.id=5867ea6696b441ac9ea74d0733b6393d&scope.project.id=a667e49b60584e6382b48fec8d858ac4&role.id=d6fe1aca91e74aad9907c855501f588f => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:41.893542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0da38-2c73-4687-a6a5-820ec83a8d95 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:41.935266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eb58e-1bdd-4593-8919-9c467ae9adc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:41.935835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb58e-1bdd-4593-8919-9c467ae9adc5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.936120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb58e-1bdd-4593-8919-9c467ae9adc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.936395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb58e-1bdd-4593-8919-9c467ae9adc5 None admin] PATH_INFO: `/v3/groups/28c61d4c046b4723b25b87a383f65245` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.937577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eb58e-1bdd-4593-8919-9c467ae9adc5 None admin] RBAC: Authorizing `identity:get_group(group_id=28c61d4c046b4723b25b87a383f65245)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:41.939097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eb58e-1bdd-4593-8919-9c467ae9adc5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:41.941132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 227/454] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/groups/28c61d4c046b4723b25b87a383f65245 => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:41.947970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:41.981665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:42.054791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fb47e-dc6e-442e-b3ad-8fd010848c6c None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:42.091009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5acae-ef7e-4b9e-9b03-05140a4e2fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:42.091621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5acae-ef7e-4b9e-9b03-05140a4e2fac None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:42.091913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5acae-ef7e-4b9e-9b03-05140a4e2fac None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:42.094879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5acae-ef7e-4b9e-9b03-05140a4e2fac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:42.094879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5acae-ef7e-4b9e-9b03-05140a4e2fac None admin] RBAC: Adding query filter params (group.id=28c61d4c046b4723b25b87a383f65245, role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=74aebe409e204672bb76c767e1d15f99) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:42.094879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5acae-ef7e-4b9e-9b03-05140a4e2fac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:42.094879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5acae-ef7e-4b9e-9b03-05140a4e2fac None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:42.100846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 228/455] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 17:29:41 2022] GET /identity/v3/role_assignments?group.id=28c61d4c046b4723b25b87a383f65245&scope.project.id=74aebe409e204672bb76c767e1d15f99&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:44.582910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa7f07-875b-40b1-bb40-acd9b4ee9d1e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:44.583758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa7f07-875b-40b1-bb40-acd9b4ee9d1e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:44.584218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa7f07-875b-40b1-bb40-acd9b4ee9d1e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:44.586728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 228/456] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:44.592199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2caa-d953-46b8-81f7-ba2eafad4b52 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:44.592981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2caa-d953-46b8-81f7-ba2eafad4b52 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:44.593395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2caa-d953-46b8-81f7-ba2eafad4b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:44.622898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-bf7f2caa-d953-46b8-81f7-ba2eafad4b52 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:44.624372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bf7f2caa-d953-46b8-81f7-ba2eafad4b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:44.719536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 229/457] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:44.732822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2382817-cfac-4f37-9e59-36cb49e45ed1 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:44.733296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2382817-cfac-4f37-9e59-36cb49e45ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:44.733710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2382817-cfac-4f37-9e59-36cb49e45ed1 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:44.735225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 229/458] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:44.740574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d8e3b-98df-443e-a421-2f36f5599108 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:44.741034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d8e3b-98df-443e-a421-2f36f5599108 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:44.741473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d8e3b-98df-443e-a421-2f36f5599108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:44.763930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a24d8e3b-98df-443e-a421-2f36f5599108 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:44.765379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a24d8e3b-98df-443e-a421-2f36f5599108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:44.793787 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:44.859470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 230/459] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:44.863696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417e7e-270f-45c2-8559-855092026bdb None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:44.864168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417e7e-270f-45c2-8559-855092026bdb None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:44.864597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417e7e-270f-45c2-8559-855092026bdb None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:44.866898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 230/460] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:44.871796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866e04f-3bf4-4c31-9e1b-151d5f7cfc1b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:44.872085 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866e04f-3bf4-4c31-9e1b-151d5f7cfc1b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:44.872358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866e04f-3bf4-4c31-9e1b-151d5f7cfc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:44.902961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d866e04f-3bf4-4c31-9e1b-151d5f7cfc1b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:44.904233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d866e04f-3bf4-4c31-9e1b-151d5f7cfc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:45.001900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 231/461] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:45.073265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de417e7e-270f-45c2-8559-855092026bdb None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.204664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de417e7e-270f-45c2-8559-855092026bdb None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.221420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.247727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.252560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.252850 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.253129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.253980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e63ad-aca4-4895-ba39-c6065ce7791b None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.254267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e63ad-aca4-4895-ba39-c6065ce7791b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.254541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e63ad-aca4-4895-ba39-c6065ce7791b None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.257748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 232/462] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:45.261978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1412c4b-2d5a-4494-bc09-72f35e797c52 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.262511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1412c4b-2d5a-4494-bc09-72f35e797c52 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.263016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1412c4b-2d5a-4494-bc09-72f35e797c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.265896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.267690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.286061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c1412c4b-2d5a-4494-bc09-72f35e797c52 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:45.287514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c1412c4b-2d5a-4494-bc09-72f35e797c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:45.295527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:45.296406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:45.297236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:45.297236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:45.297236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:45.299044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 231/463] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:45.300555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.378011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 233/464] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:29:45.378011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1412c4b-2d5a-4494-bc09-72f35e797c52 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.399323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.423482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadcb87-c92c-463d-bb79-ad24ff8c2575 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.465050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.465695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.466008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.466280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.478389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.480044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:45.497757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:45.498994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:45.500069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:45.500069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:45.500069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:45.501825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 232/465] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:45.502670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.509013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1412c4b-2d5a-4494-bc09-72f35e797c52 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.526743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.553036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.553683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.554007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.554286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.555455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:45.555790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.558271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.577988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 234/466] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:45.579166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.633912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cabe9-57a6-41a6-9bf7-d2b22411c5ff None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.671935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.672991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.673482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.674016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.686299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.688074 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.691511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:45.705695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:45.708312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:45.709158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:45.709158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:45.709158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:45.709158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 233/467] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:45.709158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.723155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61420c2-3372-4210-9dfb-4827688d8522 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.762909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.763799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.764078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.764353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.765052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:45.765387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.766793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.781228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 235/468] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:45.782239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.791332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.827767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa58d3-6ceb-48b8-9236-39851899e18b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.867576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548d1db-a941-4243-97d6-e25091c879c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.868646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548d1db-a941-4243-97d6-e25091c879c1 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.868971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548d1db-a941-4243-97d6-e25091c879c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.869250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548d1db-a941-4243-97d6-e25091c879c1 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.869921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548d1db-a941-4243-97d6-e25091c879c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.872423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548d1db-a941-4243-97d6-e25091c879c1 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.904849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b613f5-bd40-46ac-9e3c-e4a6cc0dc0f3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.944046 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.945105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.945412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.945732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.948522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.956504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.958012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.965013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 234/469] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:45 2022] POST /identity/v3/users => generated 246 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:45.970952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548d1db-a941-4243-97d6-e25091c879c1 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:45.973878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:45.974789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:45.976337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:45.976337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:45.976337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:45.976337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 236/470] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:45.981777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.117282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44035d90-e452-4fb0-ae8c-04724922f521 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.134458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548d1db-a941-4243-97d6-e25091c879c1 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.159001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb43828-2dfe-423d-8d14-83919364fc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.159340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb43828-2dfe-423d-8d14-83919364fc0f None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.159617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb43828-2dfe-423d-8d14-83919364fc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.159890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb43828-2dfe-423d-8d14-83919364fc0f None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.160530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb43828-2dfe-423d-8d14-83919364fc0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.161989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb43828-2dfe-423d-8d14-83919364fc0f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.165828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:46.175416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.176065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.176323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.176606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.180820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:46.180820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.180820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.192595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 235/471] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:46.193446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.268514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 237/472] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:45 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:46.270267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b5ae-3b22-4398-a2b8-a0e7110b463c None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.270569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b5ae-3b22-4398-a2b8-a0e7110b463c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.270847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b5ae-3b22-4398-a2b8-a0e7110b463c None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.273279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 238/473] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:46.274208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6b5ae-3b22-4398-a2b8-a0e7110b463c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.308754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd293348-86c4-4764-bd91-ec21337eff3a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.345528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:46.349218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e4363-2b67-4d66-aac5-a968162d675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.350114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4363-2b67-4d66-aac5-a968162d675b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.351152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4363-2b67-4d66-aac5-a968162d675b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.351605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4363-2b67-4d66-aac5-a968162d675b None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.352594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e4363-2b67-4d66-aac5-a968162d675b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:46.353038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e4363-2b67-4d66-aac5-a968162d675b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.354642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e4363-2b67-4d66-aac5-a968162d675b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.371521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 236/474] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:46.373012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661bb53-e685-40a4-8229-1d18b142d1d7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.373455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661bb53-e685-40a4-8229-1d18b142d1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.373875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661bb53-e685-40a4-8229-1d18b142d1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.394712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6661bb53-e685-40a4-8229-1d18b142d1d7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:46.396413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6661bb53-e685-40a4-8229-1d18b142d1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:46.434007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6b5ae-3b22-4398-a2b8-a0e7110b463c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.451686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:46.479772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70c614-9dde-4b44-9198-146047226fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.480837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70c614-9dde-4b44-9198-146047226fd8 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.481118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70c614-9dde-4b44-9198-146047226fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.481531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70c614-9dde-4b44-9198-146047226fd8 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.482244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70c614-9dde-4b44-9198-146047226fd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.483646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70c614-9dde-4b44-9198-146047226fd8 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.484942 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 237/475] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:46.485596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661bb53-e685-40a4-8229-1d18b142d1d7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.598822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 239/476] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:46 2022] POST /identity/v3/users => generated 248 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:46.609519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661bb53-e685-40a4-8229-1d18b142d1d7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.651143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9f399-fafa-4fdb-8c04-92942ed0accc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.651764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9f399-fafa-4fdb-8c04-92942ed0accc None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.652045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9f399-fafa-4fdb-8c04-92942ed0accc None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.652375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9f399-fafa-4fdb-8c04-92942ed0accc None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.653036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9f399-fafa-4fdb-8c04-92942ed0accc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.654646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9f399-fafa-4fdb-8c04-92942ed0accc None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.770077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 238/477] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:46 2022] POST /identity/v3/users => generated 249 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:46.791560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70c614-9dde-4b44-9198-146047226fd8 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.909924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:46.915993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70c614-9dde-4b44-9198-146047226fd8 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.957735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.958794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.959259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.959707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.960763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.962516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.998142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.sql.core [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:29:46.998142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 17:29:46.998142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [parameters: {'id': '2a7b20b93d794593a062209e86b4935d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 17:29:46.998142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81886) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application result.read() Aug 24 17:29:47.015931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:29:47.017036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application yield resource Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise self.value Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application session.commit() Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:29:47.018252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:29:47.019384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:29:47.020472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application result.read() Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:29:47.021561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [parameters: {'id': '2a7b20b93d794593a062209e86b4935d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:47.022585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 17:29:47.024383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:47.026587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 240/478] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 17:29:46 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:47.030813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9f399-fafa-4fdb-8c04-92942ed0accc None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:47.072234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:47.153015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9f399-fafa-4fdb-8c04-92942ed0accc None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:47.190122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240daa1e-6521-4b15-9b9b-c5916a3b6546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:47.190680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240daa1e-6521-4b15-9b9b-c5916a3b6546 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:47.190961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240daa1e-6521-4b15-9b9b-c5916a3b6546 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:47.191236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240daa1e-6521-4b15-9b9b-c5916a3b6546 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:47.206268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240daa1e-6521-4b15-9b9b-c5916a3b6546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-240daa1e-6521-4b15-9b9b-c5916a3b6546 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:47.213090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:47.215175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:47.217225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 239/479] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:29:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:47.223664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:47.382357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55e20e-b48b-444d-9934-14902112b98d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:47.432438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88830e9d-0a24-4df0-a121-b0872196c868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:47.433392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88830e9d-0a24-4df0-a121-b0872196c868 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:47.433912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88830e9d-0a24-4df0-a121-b0872196c868 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:47.434387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88830e9d-0a24-4df0-a121-b0872196c868 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:47.435274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88830e9d-0a24-4df0-a121-b0872196c868 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:47.435775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88830e9d-0a24-4df0-a121-b0872196c868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:47.437360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88830e9d-0a24-4df0-a121-b0872196c868 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:47.452375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 241/480] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:48.161475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7681f-4c7e-4039-87f4-c2ad2cc9d6af None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.161993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7681f-4c7e-4039-87f4-c2ad2cc9d6af None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.162095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7681f-4c7e-4039-87f4-c2ad2cc9d6af None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.163801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 240/481] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:48.169044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0262b9d-1da9-4e43-ad28-9cc3a446b382 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.169875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0262b9d-1da9-4e43-ad28-9cc3a446b382 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.170450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0262b9d-1da9-4e43-ad28-9cc3a446b382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.198908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-e0262b9d-1da9-4e43-ad28-9cc3a446b382 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:48.200477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e0262b9d-1da9-4e43-ad28-9cc3a446b382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:48.236263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:48.310279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 242/482] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:48.660318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9174c-e1bb-4df7-adf1-9201ffdd4805 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.660687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9174c-e1bb-4df7-adf1-9201ffdd4805 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.660865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9174c-e1bb-4df7-adf1-9201ffdd4805 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.662554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 241/483] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:48.668618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0262b9d-1da9-4e43-ad28-9cc3a446b382 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:48.670551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0ce7-fc00-44db-a399-2d2a428dc14e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.670835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0ce7-fc00-44db-a399-2d2a428dc14e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.671223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0ce7-fc00-44db-a399-2d2a428dc14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.697732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-85ec0ce7-fc00-44db-a399-2d2a428dc14e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:48.698965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-85ec0ce7-fc00-44db-a399-2d2a428dc14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:48.775036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0262b9d-1da9-4e43-ad28-9cc3a446b382 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:48.795189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 242/484] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:48.815644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e5f33-5329-46df-977d-98a06bc7d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:48.816593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e5f33-5329-46df-977d-98a06bc7d7dd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.817040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e5f33-5329-46df-977d-98a06bc7d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.817492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e5f33-5329-46df-977d-98a06bc7d7dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.835329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e5f33-5329-46df-977d-98a06bc7d7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-394e5f33-5329-46df-977d-98a06bc7d7dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:48.840233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:48.841100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:48.841900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:48.845364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 243/485] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:48.850967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec0ce7-fc00-44db-a399-2d2a428dc14e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:48.895936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:48.976720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec0ce7-fc00-44db-a399-2d2a428dc14e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.013281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.014569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.015885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.016356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.017046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:49.017351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.018747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.028002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 243/486] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:49.033194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e5f33-5329-46df-977d-98a06bc7d7dd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.061750 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.155186 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.216252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e5f33-5329-46df-977d-98a06bc7d7dd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.286849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b19ff-9b7b-4d92-b441-d9d693a81531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.287488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b19ff-9b7b-4d92-b441-d9d693a81531 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.287768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b19ff-9b7b-4d92-b441-d9d693a81531 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.288103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b19ff-9b7b-4d92-b441-d9d693a81531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.301708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.304090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b19ff-9b7b-4d92-b441-d9d693a81531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.305886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b19ff-9b7b-4d92-b441-d9d693a81531 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-b27b19ff-9b7b-4d92-b441-d9d693a81531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:49.326614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:49.327626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:49.328904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:49.328904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:49.328904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:49.329082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83edc5b-ff0b-4a65-a6cc-1565d8f6561b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.329164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 244/487] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:49.329428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5a922-72bb-488f-832c-8601e32ece12 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.330154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5a922-72bb-488f-832c-8601e32ece12 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.330154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5a922-72bb-488f-832c-8601e32ece12 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.331527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 245/488] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:49.336276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5a922-72bb-488f-832c-8601e32ece12 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.378644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e73263-51f5-436d-b49a-f487c6bf836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.379282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e73263-51f5-436d-b49a-f487c6bf836e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.379564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e73263-51f5-436d-b49a-f487c6bf836e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.379839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e73263-51f5-436d-b49a-f487c6bf836e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.394343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e73263-51f5-436d-b49a-f487c6bf836e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-36e73263-51f5-436d-b49a-f487c6bf836e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:49.402137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:49.403103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:49.403916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:49.405942 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 244/489] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:49.407358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add32df-08c8-4e4b-86b0-237ca8a85a54 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.407822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add32df-08c8-4e4b-86b0-237ca8a85a54 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.408330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add32df-08c8-4e4b-86b0-237ca8a85a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.431896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.434054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-8add32df-08c8-4e4b-86b0-237ca8a85a54 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:49.435289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8add32df-08c8-4e4b-86b0-237ca8a85a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:49.470604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5a922-72bb-488f-832c-8601e32ece12 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.507778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 245/490] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:49.508452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add32df-08c8-4e4b-86b0-237ca8a85a54 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.517246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39194bd2-867c-49c7-adc6-ec99d727f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.518516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194bd2-867c-49c7-adc6-ec99d727f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.519038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194bd2-867c-49c7-adc6-ec99d727f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.519511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194bd2-867c-49c7-adc6-ec99d727f5b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.520684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39194bd2-867c-49c7-adc6-ec99d727f5b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:49.521529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39194bd2-867c-49c7-adc6-ec99d727f5b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.523556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39194bd2-867c-49c7-adc6-ec99d727f5b8 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.540740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 246/491] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:49.542456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1434a-5e58-482c-8bcb-7a7251554bdd None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.542947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1434a-5e58-482c-8bcb-7a7251554bdd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.543429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1434a-5e58-482c-8bcb-7a7251554bdd None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.545439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 247/492] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:49.547160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1434a-5e58-482c-8bcb-7a7251554bdd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.599228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.627860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add32df-08c8-4e4b-86b0-237ca8a85a54 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.667992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b124cdf-b0b8-4ab6-8090-0ce1415e7de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.668605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b124cdf-b0b8-4ab6-8090-0ce1415e7de6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.668938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b124cdf-b0b8-4ab6-8090-0ce1415e7de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.669155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b124cdf-b0b8-4ab6-8090-0ce1415e7de6 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.669838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b124cdf-b0b8-4ab6-8090-0ce1415e7de6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:49.670184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b124cdf-b0b8-4ab6-8090-0ce1415e7de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.671726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b124cdf-b0b8-4ab6-8090-0ce1415e7de6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.686107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 246/493] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:49.686107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae53445-f27e-4956-81b4-9253c1772fb7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.686107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae53445-f27e-4956-81b4-9253c1772fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.686107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae53445-f27e-4956-81b4-9253c1772fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.688781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1434a-5e58-482c-8bcb-7a7251554bdd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.705424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-3ae53445-f27e-4956-81b4-9253c1772fb7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:49.706817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3ae53445-f27e-4956-81b4-9253c1772fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:49.728170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.740567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.742136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.742710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.743276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.752743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.755438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:49.774410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:49.775358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:49.776281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 248/494] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:49.776852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.789225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 247/495] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:49.790128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae53445-f27e-4956-81b4-9253c1772fb7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.871388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.920310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477ba34-6dcb-4ae0-a22e-048c77dd7a00 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.931385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae53445-f27e-4956-81b4-9253c1772fb7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.959641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.960365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.960647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.960921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.972788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.975243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.975243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.975243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.976013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.976100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.976386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:49.976943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.978740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:49.989639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:49.990605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:49.992492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:49.992492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:49.992492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:49.992492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 249/496] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:49.992492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.062257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 248/497] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:50.062919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.091032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:50.154034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10159d-ddf0-4cf8-a23c-c1a50fd2f7a3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.179289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37e6d9-945b-49f4-a314-7dfeb3e1a02a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.189628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362a9bf-136f-4a93-a2a8-3540ccfcb944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.190295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362a9bf-136f-4a93-a2a8-3540ccfcb944 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.190582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362a9bf-136f-4a93-a2a8-3540ccfcb944 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.190854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362a9bf-136f-4a93-a2a8-3540ccfcb944 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.201206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362a9bf-136f-4a93-a2a8-3540ccfcb944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:50.203885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-a362a9bf-136f-4a93-a2a8-3540ccfcb944 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:50.206378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:50.207371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:50.208172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 250/498] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:50.208812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362a9bf-136f-4a93-a2a8-3540ccfcb944 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.216599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.217206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.217485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.217758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.218440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:50.218744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:50.220203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:50.232620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 249/499] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:50.233421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.308795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:50.311658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362a9bf-136f-4a93-a2a8-3540ccfcb944 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.343359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8c281-f886-4989-a0a2-9137c3847902 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.346754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94d441-93ad-4532-ae7f-ac163b194e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.347378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94d441-93ad-4532-ae7f-ac163b194e59 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.347658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94d441-93ad-4532-ae7f-ac163b194e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.347928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94d441-93ad-4532-ae7f-ac163b194e59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.359580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94d441-93ad-4532-ae7f-ac163b194e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-4f94d441-93ad-4532-ae7f-ac163b194e59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:50.364686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:50.365713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 251/500] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:50.367042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99abd8e-bd90-4722-ac35-f793a8c9c62d None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.368065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99abd8e-bd90-4722-ac35-f793a8c9c62d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.368065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99abd8e-bd90-4722-ac35-f793a8c9c62d None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.368839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 252/501] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:50.369541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99abd8e-bd90-4722-ac35-f793a8c9c62d None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.390296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea683e3-418b-45c0-b298-4ea7937c88d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.391136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea683e3-418b-45c0-b298-4ea7937c88d2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.391253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea683e3-418b-45c0-b298-4ea7937c88d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.391435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea683e3-418b-45c0-b298-4ea7937c88d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.449001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:50.465994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea683e3-418b-45c0-b298-4ea7937c88d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:50.467487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea683e3-418b-45c0-b298-4ea7937c88d2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:50.483020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99abd8e-bd90-4722-ac35-f793a8c9c62d None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.519061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.519668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.519954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.520230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.520911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:50.521214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:50.523340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:50.531696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 253/502] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:50.532540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.558191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:50.580952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 250/503] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:50.581866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea683e3-418b-45c0-b298-4ea7937c88d2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.635553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129c8de-3055-43cf-a3a9-063fd09300d3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.666636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:50.671196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c537fe8-5c27-41d2-a9d4-776c0c5a0f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.671827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c537fe8-5c27-41d2-a9d4-776c0c5a0f7d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.672122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c537fe8-5c27-41d2-a9d4-776c0c5a0f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.672379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c537fe8-5c27-41d2-a9d4-776c0c5a0f7d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.680704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c537fe8-5c27-41d2-a9d4-776c0c5a0f7d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:50.682169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c537fe8-5c27-41d2-a9d4-776c0c5a0f7d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:50.683846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea683e3-418b-45c0-b298-4ea7937c88d2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-0c537fe8-5c27-41d2-a9d4-776c0c5a0f7d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:50.694665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:50.695581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:50.696458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 254/504] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:50.697465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8d75-7b79-488f-826c-4b94809940da None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.697690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8d75-7b79-488f-826c-4b94809940da None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.697999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8d75-7b79-488f-826c-4b94809940da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.716263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b2ee8d75-7b79-488f-826c-4b94809940da None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:50.717524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b2ee8d75-7b79-488f-826c-4b94809940da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:50.723944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.724547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.724835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.725103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.726355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:50.726657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:50.728035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:50.736272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 251/505] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:50.737010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.787097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 255/506] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:50.788511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee8d75-7b79-488f-826c-4b94809940da None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.803183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:50.840826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e536c15-f19e-4f1d-bbfa-fb836c8c94cf None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.877160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.877796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.878127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.878444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.889568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:50.891052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:50.892372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee8d75-7b79-488f-826c-4b94809940da None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:50.899153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:50.911231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:50.918176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:50.919035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:50.919035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:50.919035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:50.919035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 252/507] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:50.919035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:50.938263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:50.938897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:50.939431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:50.939705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:50.950493 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:50.951952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:50.967567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:50.974193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:50.975055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:50.975055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:50.975055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:50.975055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 256/508] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:50.975055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.032299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98289c78-f337-47cc-a6da-0cd16302181e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.038615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.069711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.070456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.070745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.071023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.071765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:51.072072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.073521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.079968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fff044-346c-4506-83fb-228fa859ae3c None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.091401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 253/509] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:51.092214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.116128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.116903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.117042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.117316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.128740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.130236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:51.144358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:51.145340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:51.146222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:51.146222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:51.146222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:51.146222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 257/510] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:51.146222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.151368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.196459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa8cc5-1f40-43f5-8b9d-54dce5efbd05 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.229926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.230593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.230875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.231151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.231796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:51.232104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.233540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.245527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 254/511] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:51.246365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.251620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27def149-ace1-4b0d-90a4-d148bfbdb44a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.268074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.285097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.285630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.286003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.286280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.286919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:51.287220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.288649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.300452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 258/512] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:51.301784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.352104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682593a7-e2b5-4ff3-bdcc-3a6ae7c7578c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.368918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.387450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.388079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.388363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.388673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.399858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.401321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.402829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857af8-4158-4032-8cfd-1cd0e93088f7 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:51.415868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:51.416804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:51.417645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:51.417645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:51.417645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:51.417645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 255/513] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:51.417884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.443552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e3285-f7fc-47f3-8cfb-0e0e79808114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.445158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3285-f7fc-47f3-8cfb-0e0e79808114 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.445424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3285-f7fc-47f3-8cfb-0e0e79808114 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.445694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3285-f7fc-47f3-8cfb-0e0e79808114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.492949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.516985 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e3285-f7fc-47f3-8cfb-0e0e79808114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.518710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e3285-f7fc-47f3-8cfb-0e0e79808114 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.519764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e09d72-9035-4da4-9e3f-3fc0c44bdaf2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.555013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.555528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.555802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.556089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.556703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:51.557000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.558443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.570667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 256/514] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:51.571498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.599023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.624901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 259/515] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:51.625745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e3285-f7fc-47f3-8cfb-0e0e79808114 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.671515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546bb4d-d7d4-4535-9f92-fcb0222a36df None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.705001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.706519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.707153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.707434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.707707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.715045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.717994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.728305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e3285-f7fc-47f3-8cfb-0e0e79808114 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:51.730317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:51.731376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:51.732294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 257/516] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:51.732969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.765083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.765718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.766031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.766306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.773412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.774901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:29:51.786710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:51.787691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:51.789134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 260/517] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:51.789724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.826945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.839370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7215a3-237a-43eb-a0d0-b5aa6be3e961 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.873878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.874482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.874764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.875036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.875680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:51.875986 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.877393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.889586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 258/518] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:51.890543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:51.898580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e365b6-dd52-4732-b2d1-46cc7552552d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:51.937683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:51.938350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:51.938624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:51.939143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:51.942135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:51.947185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:51.948636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:51.960753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:51.961664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 261/519] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:51.962963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.001078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7255e1f-e996-4112-8ffd-16f270a5eae9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.036918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.037568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.038451 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.038727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.049383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.050887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:52.059822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:52.064558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0acc64-eac2-4b4b-9d00-bdfabcb0a1b9 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:52.066399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:52.067556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:52.068420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:52.068420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:52.068420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:52.068420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 259/520] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:52.068420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.101620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.102180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.102460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.102731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.103456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:52.103761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.105177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:52.141045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 262/521] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 24 17:29:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:52.141885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.174380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6c955-d49c-4707-b2d0-f7ebe4e8df67 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.201299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:52.208107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.213173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.213173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.213173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.213173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:52.213173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.213173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:52.230284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 260/522] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:52.231116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.248552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd6b7-85a3-4db6-8244-ed2ed73cf451 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.285219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9edc8-41e2-4e03-a790-13254c4b21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.285825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9edc8-41e2-4e03-a790-13254c4b21fa None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.286142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9edc8-41e2-4e03-a790-13254c4b21fa None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.286445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9edc8-41e2-4e03-a790-13254c4b21fa None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.287088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9edc8-41e2-4e03-a790-13254c4b21fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.288645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9edc8-41e2-4e03-a790-13254c4b21fa None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:52.293923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:52.335468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7c7ac-7add-4a08-b4b6-e385fca38035 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.370163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.370739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.370974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.371249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.371978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:52.372287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.373755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:52.391018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 261/523] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 17:29:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:52.391887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.420311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 263/524] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:52 2022] POST /identity/v3/users => generated 247 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:52.421245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9edc8-41e2-4e03-a790-13254c4b21fa None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.507587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d270d-79dc-4794-8293-e4f55e7a1996 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.528107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:52.540103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9edc8-41e2-4e03-a790-13254c4b21fa None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.549456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.550109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.550395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.550753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.551419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:52.551803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.552279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:52.553236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:52.569052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 262/525] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:52.570198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.579115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e373485-2565-42d2-8852-1c35702293be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.579554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e373485-2565-42d2-8852-1c35702293be None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.579851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e373485-2565-42d2-8852-1c35702293be None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.580134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e373485-2565-42d2-8852-1c35702293be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.660031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e373485-2565-42d2-8852-1c35702293be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.661564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e373485-2565-42d2-8852-1c35702293be None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:52.687555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9027162-9592-4bea-bffa-13928bf3c95e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.725575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.726214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.726492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.726775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.727640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=20dd03dd2fa04147bba827f983a58034) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:52.727811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.729265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:52.741914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 263/526] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:52 2022] GET /identity/v3/role_assignments?user.id=20dd03dd2fa04147bba827f983a58034&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:52.741914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.787080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:52.795335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 264/527] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:52.796550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e373485-2565-42d2-8852-1c35702293be None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:52.814226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:52.878385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40025260-e314-4628-8e91-cfbecda3c661 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.935507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e373485-2565-42d2-8852-1c35702293be None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:52.941444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1adfd-cc79-46c7-865c-d0cf78ef55d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.942090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1adfd-cc79-46c7-865c-d0cf78ef55d0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.942367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1adfd-cc79-46c7-865c-d0cf78ef55d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.942634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1adfd-cc79-46c7-865c-d0cf78ef55d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.978978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:52.979642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:52.980088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:52.980366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:52.987929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:52.989507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:29:53.002768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:53.003678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:53.004564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 265/528] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:53.005500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.025366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1adfd-cc79-46c7-865c-d0cf78ef55d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.027487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1adfd-cc79-46c7-865c-d0cf78ef55d0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.085046 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:53.085713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:53.122880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63553cd-228a-4879-9df2-1697bb41f6cd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:53.157865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:53.158507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:53.158796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:53.159142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:53.159534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 264/529] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:53.160661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1adfd-cc79-46c7-865c-d0cf78ef55d0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.170749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.171815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:53.187294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:53.188432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:53.189740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:53.189740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:53.189740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:53.193293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 266/530] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:53.195054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.279400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1adfd-cc79-46c7-865c-d0cf78ef55d0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:53.301815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61fb83-4016-498a-b527-5cef03bb5e25 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:53.316713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:53.317532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:53.318008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:53.318437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:53.320123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:53.320647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.322314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.326577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:53.335747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 265/531] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:53.339341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.341031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:53.345007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:53.345656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:53.345981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:53.346263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:53.358443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.359918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:53.375659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:53.376613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:53.377433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:53.377433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:53.377433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:53.377433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 267/532] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:53.378138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.456960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a88e23-e918-4665-98e9-c9a50ea7372e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:53.488591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef09155-0ac1-42a1-bcd6-553fba4215c2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:53.500203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:53.501156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:53.501727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:53.502041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:53.502702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:53.503162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.505242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.521203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 266/533] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:53.523135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.531938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:53.532550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:53.532830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:53.533098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:53.533876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=b4d874536170494496769b4f37234b2f) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:53.534268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.535861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.542735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 268/534] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/role_assignments?user.id=b4d874536170494496769b4f37234b2f&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:53.550765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.592580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:53.626337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:53.637190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838f784-4579-4a49-85b9-548212491d33 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:53.676297 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:53.676932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:53.677219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:53.677495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:53.678206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:53.679353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.680621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.694789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 267/535] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:53.700209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.745101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa564e-b438-4482-ae46-25ccde04a876 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:53.789564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:53.790363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:53.790738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:53.791107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:53.811224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.811224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:29:53.828905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:53.830963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:53.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:29:53.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:29:53.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:53.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:53.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:53.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:53.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:53.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:53.837587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 269/536] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:53.844498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:53.886964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81d0ef-b809-4edb-8346-a6f6950bb5a4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:53.922948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:53.935864 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:53.936796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:53.937237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:53.937676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:53.945413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:53.947143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:29:53.959675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:53.960773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:53.961639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:29:53.961639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:29:53.961639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:53.961639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:53.961639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:53.961639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:53.961639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:53.961639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:53.964132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 268/537] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:53.973956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:54.095157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b364e-157a-4a2d-b0a8-1f420ee390db None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:54.108364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:54.114973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d8763-f0b5-4abf-b6db-78bbbdca15eb None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:54.163364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:54.165478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:54.166822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:54.167996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:54.169078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:54.170564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:54.172474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:54.184423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:54.185372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:54.185871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:54.186345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:54.187253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:54.187775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:54.189667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:54.200577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 269/538] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:54.214271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:54.236369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 270/539] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:54.243843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:54.272992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:54.388982 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099dc7e-8220-4b03-9a80-f8f8a3dfa4a0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:54.411143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05b99b-5997-4998-b6bd-efe943aa53f0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:54.459970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbf66a-e5e2-4327-b096-173c8bac50ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:54.461470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbf66a-e5e2-4327-b096-173c8bac50ec None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:54.462747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbf66a-e5e2-4327-b096-173c8bac50ec None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:54.463873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbf66a-e5e2-4327-b096-173c8bac50ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:54.465021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbf66a-e5e2-4327-b096-173c8bac50ec None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=6c7a064cf91243648c6cc6d6ca576505) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:54.466717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbf66a-e5e2-4327-b096-173c8bac50ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:54.468933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbf66a-e5e2-4327-b096-173c8bac50ec None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:54.485885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 270/540] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:54 2022] GET /identity/v3/role_assignments?user.id=6c7a064cf91243648c6cc6d6ca576505&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:54.486105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1301-6c13-47c1-9d83-787289070928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:54.486105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1301-6c13-47c1-9d83-787289070928 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:54.486105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1301-6c13-47c1-9d83-787289070928 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:54.486105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1301-6c13-47c1-9d83-787289070928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:54.486393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb1301-6c13-47c1-9d83-787289070928 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=efc068011c14492d9ba483650cf632e5) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:54.486393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb1301-6c13-47c1-9d83-787289070928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:54.487983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb1301-6c13-47c1-9d83-787289070928 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:54.509211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 271/541] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:54 2022] GET /identity/v3/role_assignments?user.id=efc068011c14492d9ba483650cf632e5&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:54.566808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d4c0-9747-47c1-9755-8b5fccf02475 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:54.567079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d4c0-9747-47c1-9755-8b5fccf02475 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:54.567358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d4c0-9747-47c1-9755-8b5fccf02475 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:54.569638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 271/542] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:54.575557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5760431-0dc6-438f-865f-904ea8d7a02d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:54.575884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5760431-0dc6-438f-865f-904ea8d7a02d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:54.576106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5760431-0dc6-438f-865f-904ea8d7a02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:54.603900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a5760431-0dc6-438f-865f-904ea8d7a02d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:54.605184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a5760431-0dc6-438f-865f-904ea8d7a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:54.633034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:54.700971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 272/543] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:54.980732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b59cb-fc6a-4165-a999-811fbe2a2678 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:54.981063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b59cb-fc6a-4165-a999-811fbe2a2678 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:54.981288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b59cb-fc6a-4165-a999-811fbe2a2678 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:54.982663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 272/544] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:54.999586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d203f47-5d90-45fa-88d0-bf170f3596eb None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:54.999877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d203f47-5d90-45fa-88d0-bf170f3596eb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.000159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d203f47-5d90-45fa-88d0-bf170f3596eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:55.012878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b59cb-fc6a-4165-a999-811fbe2a2678 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:55.029982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7d203f47-5d90-45fa-88d0-bf170f3596eb None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:55.031191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7d203f47-5d90-45fa-88d0-bf170f3596eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:55.160140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 273/545] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:55.217620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b59cb-fc6a-4165-a999-811fbe2a2678 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:55.228991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:55.274848 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a8617-d8d5-4051-8861-e7a1bcb2d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:55.275399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8617-d8d5-4051-8861-e7a1bcb2d696 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:55.275680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8617-d8d5-4051-8861-e7a1bcb2d696 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.275954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8617-d8d5-4051-8861-e7a1bcb2d696 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:55.292725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a8617-d8d5-4051-8861-e7a1bcb2d696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-342a8617-d8d5-4051-8861-e7a1bcb2d696 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:55.297744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:55.298654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:55.299443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:55.302033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 273/546] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:55.308105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d203f47-5d90-45fa-88d0-bf170f3596eb None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:55.424564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:55.446769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d203f47-5d90-45fa-88d0-bf170f3596eb None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:55.494860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:55.495659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:55.496305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.496616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:55.497297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:55.497605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:55.499032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:55.514898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 274/547] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:55.520888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a8617-d8d5-4051-8861-e7a1bcb2d696 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:55.551796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:55.680932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0647d-7277-439c-9aed-38b72960e8f4 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:55.721960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b76ddf-dc94-41ff-ba06-e6e6106e2861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:55.722976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b76ddf-dc94-41ff-ba06-e6e6106e2861 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:55.723460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b76ddf-dc94-41ff-ba06-e6e6106e2861 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.723954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b76ddf-dc94-41ff-ba06-e6e6106e2861 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:55.736500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b76ddf-dc94-41ff-ba06-e6e6106e2861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-96b76ddf-dc94-41ff-ba06-e6e6106e2861 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:55.742760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:55.743747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:55.744541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:55.747860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 275/548] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:55.753847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b76ddf-dc94-41ff-ba06-e6e6106e2861 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:55.758877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:55.759288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a8617-d8d5-4051-8861-e7a1bcb2d696 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:55.822900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:55.823446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:55.823719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.823993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:55.839934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:55.841772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:55.872408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:55.873384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:55.874293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:55.874293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:55.874293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:55.880844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 274/549] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:55.889033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:55.933271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b76ddf-dc94-41ff-ba06-e6e6106e2861 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:55.942217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:55.979566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d63b2c-d203-49af-b066-0216e0541726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:55.980199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d63b2c-d203-49af-b066-0216e0541726 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:55.980534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d63b2c-d203-49af-b066-0216e0541726 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.980817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d63b2c-d203-49af-b066-0216e0541726 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:55.981512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d63b2c-d203-49af-b066-0216e0541726 None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:55.981828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d63b2c-d203-49af-b066-0216e0541726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:55.983333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d63b2c-d203-49af-b066-0216e0541726 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:55.994478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 276/550] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:55.995770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c65f1-c7e8-487a-a93b-873414f83d72 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:55.996060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c65f1-c7e8-487a-a93b-873414f83d72 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.996339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c65f1-c7e8-487a-a93b-873414f83d72 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.002325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 277/551] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:56.003095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c65f1-c7e8-487a-a93b-873414f83d72 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.049416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719bff9-4b66-41b9-bd9a-c9f66630e9e3 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.089272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c9aa-01ef-4a01-ac0e-630aeead03a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.090549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c9aa-01ef-4a01-ac0e-630aeead03a9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.091023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c9aa-01ef-4a01-ac0e-630aeead03a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.091469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c9aa-01ef-4a01-ac0e-630aeead03a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.092352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027c9aa-01ef-4a01-ac0e-630aeead03a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:56.093231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027c9aa-01ef-4a01-ac0e-630aeead03a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:56.095100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027c9aa-01ef-4a01-ac0e-630aeead03a9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:56.110156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:56.111621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 275/552] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:56.113783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00ee25-5da3-4d9b-b3b3-7b577d2ef748 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.114270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00ee25-5da3-4d9b-b3b3-7b577d2ef748 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.114717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00ee25-5da3-4d9b-b3b3-7b577d2ef748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.137381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7c00ee25-5da3-4d9b-b3b3-7b577d2ef748 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:56.138921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7c00ee25-5da3-4d9b-b3b3-7b577d2ef748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:56.156600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c65f1-c7e8-487a-a93b-873414f83d72 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.195802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.196723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.197201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.197644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.208671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:56.211177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:56.229606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:56.230700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:56.231618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:56.231618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:56.231618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:56.231793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 276/553] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:56.231793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00ee25-5da3-4d9b-b3b3-7b577d2ef748 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.233791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 278/554] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:56.238941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.251117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:56.358957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6c75c-2a0d-4f08-937a-52e81759f895 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.377817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:56.398412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.399978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.400438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.401152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.402465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:56.402954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:56.404596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:56.421077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 279/555] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:56.423461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00ee25-5da3-4d9b-b3b3-7b577d2ef748 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.426282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.495787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.496441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.496723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.497050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.515254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:56.517952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:56.531520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:56.532894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:29:56.533886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:56.536890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 277/556] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:56.541891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.563332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbca60-7e51-4d3f-a4a4-b9181af114e5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.630306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.630940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.631293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.631615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] PATH_INFO: `/v3/users/nova` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.650234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:56.651813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:56.669040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:56.672004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:56.672893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:29:56.674273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:56.682528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 280/557] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:56.701207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.709319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf60d3-4c10-4559-b4c7-d63102102bf6 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.731988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:56.751488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.752340 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.752798 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.753207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.754130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:56.754584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:56.757010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:56.777661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 278/558] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:56.783047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.797660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34654b0-a7f9-4939-af6c-8f67c32235de None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.835082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147164c6-7c14-4ab1-b23f-c98712265145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.835976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147164c6-7c14-4ab1-b23f-c98712265145 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.836257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147164c6-7c14-4ab1-b23f-c98712265145 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.836533 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147164c6-7c14-4ab1-b23f-c98712265145 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.837263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147164c6-7c14-4ab1-b23f-c98712265145 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:56.837577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147164c6-7c14-4ab1-b23f-c98712265145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:56.839024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147164c6-7c14-4ab1-b23f-c98712265145 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:56.861818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 281/559] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:56.861818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66d690-fd9e-41c6-a7c1-e2c55ae25c85 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.861818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66d690-fd9e-41c6-a7c1-e2c55ae25c85 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.861818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66d690-fd9e-41c6-a7c1-e2c55ae25c85 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.861818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 282/560] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:56.861818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66d690-fd9e-41c6-a7c1-e2c55ae25c85 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.901328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae5a0f-fac6-417e-8083-762b73c7c7ef None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.931077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:56.939810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c4e9-5a74-4ca8-baa2-9c7c994c63a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.940438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c4e9-5a74-4ca8-baa2-9c7c994c63a0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.940720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c4e9-5a74-4ca8-baa2-9c7c994c63a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.940993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c4e9-5a74-4ca8-baa2-9c7c994c63a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.951601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6c4e9-5a74-4ca8-baa2-9c7c994c63a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-63a6c4e9-5a74-4ca8-baa2-9c7c994c63a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:56.956372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:56.957329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:56.958210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:56.962012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 279/561] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:56.962940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c4e9-5a74-4ca8-baa2-9c7c994c63a0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:56.969608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:56.976484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66d690-fd9e-41c6-a7c1-e2c55ae25c85 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.011558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7229bd5-ba75-4b2d-a42e-cc4745a15669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.012160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229bd5-ba75-4b2d-a42e-cc4745a15669 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.012438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229bd5-ba75-4b2d-a42e-cc4745a15669 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.012711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229bd5-ba75-4b2d-a42e-cc4745a15669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.063320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c4e9-5a74-4ca8-baa2-9c7c994c63a0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.138890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a970e-74db-4532-875e-e75f87f72c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.139525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a970e-74db-4532-875e-e75f87f72c0d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.139801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a970e-74db-4532-875e-e75f87f72c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.140069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a970e-74db-4532-875e-e75f87f72c0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.153210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a970e-74db-4532-875e-e75f87f72c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:57.154808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a970e-74db-4532-875e-e75f87f72c0d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:57.169256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7229bd5-ba75-4b2d-a42e-cc4745a15669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-7c1a970e-74db-4532-875e-e75f87f72c0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:57.170417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:57.171514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:57.172562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:57.172562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:57.172562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:57.172562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 280/562] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:57.172995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cdfcb-4829-482c-928d-2b2d859b4559 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.173282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cdfcb-4829-482c-928d-2b2d859b4559 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.173561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cdfcb-4829-482c-928d-2b2d859b4559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.177004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7229bd5-ba75-4b2d-a42e-cc4745a15669 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:57.196383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a62cdfcb-4829-482c-928d-2b2d859b4559 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:57.197620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a62cdfcb-4829-482c-928d-2b2d859b4559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:57.231118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:57.275715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 281/563] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:57.277094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ce41-74f7-4af9-a6a7-627f435bd673 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.277382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ce41-74f7-4af9-a6a7-627f435bd673 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.277659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ce41-74f7-4af9-a6a7-627f435bd673 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.283308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 282/564] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:29:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:57.284091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ce41-74f7-4af9-a6a7-627f435bd673 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.290836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:57.298617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 283/565] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:29:57.299439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7229bd5-ba75-4b2d-a42e-cc4745a15669 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.425771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ce41-74f7-4af9-a6a7-627f435bd673 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.439819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7229bd5-ba75-4b2d-a42e-cc4745a15669 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.467626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b995103-86ba-404f-90b2-6e3f73bce064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.468529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b995103-86ba-404f-90b2-6e3f73bce064 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.469028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b995103-86ba-404f-90b2-6e3f73bce064 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.469302 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b995103-86ba-404f-90b2-6e3f73bce064 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.470051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b995103-86ba-404f-90b2-6e3f73bce064 None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:57.470355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b995103-86ba-404f-90b2-6e3f73bce064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:57.471722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b995103-86ba-404f-90b2-6e3f73bce064 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:57.480666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 283/566] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:57.481919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da190d5-ab13-4a4b-b78e-d9d637c54c6e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.482210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da190d5-ab13-4a4b-b78e-d9d637c54c6e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.482487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da190d5-ab13-4a4b-b78e-d9d637c54c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.486043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.486665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.486949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.487227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.487934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:57.488241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:57.489605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:57.501738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9da190d5-ab13-4a4b-b78e-d9d637c54c6e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:57.506835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 284/567] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:57.506972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9da190d5-ab13-4a4b-b78e-d9d637c54c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:57.507053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.533385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:57.566395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:57.578386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 284/568] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:29:57.579243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da190d5-ab13-4a4b-b78e-d9d637c54c6e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.622953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6f803-84a8-483a-9df7-2a8464d880b1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.658695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.659359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.659645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.659924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.670800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:57.672283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:57.686631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da190d5-ab13-4a4b-b78e-d9d637c54c6e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:57.686772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:57.687669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:57.688564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:57.688564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:57.688564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:57.688564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 285/569] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:57.689047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.729049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.729679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.729992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.730267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.740673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:57.742174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:57.759072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:57.760224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:57.761355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:57.761355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:57.761355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:57.764776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 285/570] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:57.765496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.780930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:57.808399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6d06c-5f89-4591-a895-078db452cde1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.817797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:57.845278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.845939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.846224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.846497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.854310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:57.855766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:29:57.867767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:57.868842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 286/571] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:57.869727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.879878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490365b8-87db-4cdd-9354-76de29f3f1f0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.915771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e8bb4-66af-4d5d-8f95-c191dbae1b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.916429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e8bb4-66af-4d5d-8f95-c191dbae1b42 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.916719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e8bb4-66af-4d5d-8f95-c191dbae1b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.916995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e8bb4-66af-4d5d-8f95-c191dbae1b42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.927467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e8bb4-66af-4d5d-8f95-c191dbae1b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c84e8bb4-66af-4d5d-8f95-c191dbae1b42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:57.932729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:57.933767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 286/572] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:57.934643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e8bb4-66af-4d5d-8f95-c191dbae1b42 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.980011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311664f2-ea2b-4404-8206-75c37ea5acce None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.007019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.014237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.014847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.015124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.015397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.016042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:58.016341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.017699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.030154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 287/573] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:58.031135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.038094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e8bb4-66af-4d5d-8f95-c191dbae1b42 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.061749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.071944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.072544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.072822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.073095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.073739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:58.074076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.075448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.087738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 287/574] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:58.088540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.135582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02d81c-f6d4-4ca8-ba94-b077c9052423 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.170429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945aca4-7a0b-4332-a80c-7eddc5bdf806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.171054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945aca4-7a0b-4332-a80c-7eddc5bdf806 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.171331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945aca4-7a0b-4332-a80c-7eddc5bdf806 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.171602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945aca4-7a0b-4332-a80c-7eddc5bdf806 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.181930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945aca4-7a0b-4332-a80c-7eddc5bdf806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-d945aca4-7a0b-4332-a80c-7eddc5bdf806 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:58.186614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:58.187706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 288/575] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:58.189130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945aca4-7a0b-4332-a80c-7eddc5bdf806 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.192998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c35c08-c9f9-4c5e-bb08-a38e153599b7 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.222977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.228362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.228966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.229249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.229525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.230312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:58.230687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.232133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.243868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 288/576] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:58.244712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.279181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.291568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945aca4-7a0b-4332-a80c-7eddc5bdf806 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.326053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.326660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.326934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.327208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.327943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:58.328684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.330758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.339531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 289/577] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:58.340337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.347178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93c80d-cd68-4a3a-a936-6206eaf91821 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.381088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.382495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.382775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.383042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.389628 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.391236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:29:58.402939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:58.404757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 289/578] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:58.406520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.444243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d29cf-d7e8-4ad9-8e17-4536568a3c6d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.462949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.478407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.479033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.479320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.479593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.487526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.488938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:58.500142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:58.501265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 290/579] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:58.502241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.515061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.516312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d1d2e-e611-4f6b-b0ec-bffca3b26855 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.551540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.552174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.552417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.552688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.553358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:58.553662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.555265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.563276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 290/580] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:58.564329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.609565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af46be-4be4-4877-b819-84a7044bd548 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.644059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562f367-7541-4c33-bc8d-57ac54686952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.644716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f367-7541-4c33-bc8d-57ac54686952 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.644999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f367-7541-4c33-bc8d-57ac54686952 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.645277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f367-7541-4c33-bc8d-57ac54686952 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/20dd03dd2fa04147bba827f983a58034/roles/39294eb951d24b04be9b4b5d98b36640` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.660268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562f367-7541-4c33-bc8d-57ac54686952 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=20dd03dd2fa04147bba827f983a58034, role_id=39294eb951d24b04be9b4b5d98b36640)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.661701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562f367-7541-4c33-bc8d-57ac54686952 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.671487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206114-f0a6-45d7-a39f-2b65151e6ea0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.685760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.706329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.706956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.707238 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.707512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.718071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.719545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:58.733548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:58.734561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:58.736135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:58.736135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:58.736135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:58.736135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 291/581] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:58.736135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.754858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 291/582] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:58 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/20dd03dd2fa04147bba827f983a58034/roles/39294eb951d24b04be9b4b5d98b36640 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:58.754858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562f367-7541-4c33-bc8d-57ac54686952 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.821991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.862589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90fe02-56e1-4388-a1ec-4742e9b970a9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.870528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562f367-7541-4c33-bc8d-57ac54686952 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.909142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.910255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.910568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.910842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.911577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:58.911860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.913445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.916252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.917080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.917562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.918700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.919515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:58.919825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.921301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.927767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 292/583] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:58.929972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.941341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 292/584] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:58.942239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.966748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.047489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16bb83-a10d-4c06-ace7-ef18fafb2631 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.062555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57a048-ff0a-4289-b4cd-d81c0593b437 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.082940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.086336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.087030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.087312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.087578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.102285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.102798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.103411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.103690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.103854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.103951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.104645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:59.104986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.106364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:59.120232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:59.121229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:59.122160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:59.122160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:59.122160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:59.124044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 293/585] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:59.125113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.125788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 293/586] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:59.126736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.212777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.234009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b9156-91e3-4f72-bc88-8f95ac54ff87 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.235215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c9274-3be0-41ac-97af-d7af044fa20d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.269792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.270453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.270729 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.270995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.271753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=e698e658de56495e8d501ca6dfd75623) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:59.272116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.272840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.273467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.273567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.273748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.274053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.283359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 294/587] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/role_assignments?user.id=e698e658de56495e8d501ca6dfd75623&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:59.283359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.289611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.291329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:59.312857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:59.314435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:59.315533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:59.315533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:59.315533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:29:59.318019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 294/588] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:59.319180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.357457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.445294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b09b80-6c00-4893-8cdf-12e87229cc59 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.465595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14eed6-2fd5-44cd-a290-0d0fd17356c3 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.496779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.497192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.499382 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.499941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.500440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.503143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.503870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.504260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.504849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.505434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:59.505840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.508166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.512124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.514150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.522062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 295/589] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:59.523023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:59.529794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:59.531056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:29:59.532229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:59.535961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 295/590] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:59.536919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.653759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478539c-28d1-4acd-bea0-77111f06ab46 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.666801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5febee0-c4f9-4b6d-b363-03d59cf049dd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.669567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.692488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.693054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.693395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.694779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.695538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:59.695943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.697555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.714196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.714405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 296/591] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:59.715187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.715471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.715761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.716222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.729092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.730746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:59.744143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:59.745176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:59.746206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 296/592] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:59.746787 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.790251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.851147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d377b-d32c-419e-a9c8-75f650d55749 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.863745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f6842-02e4-45a3-8502-012afb8a1dfd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.889580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.890386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.890775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.891245 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.892083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:59.892491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.894319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.908159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.908865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.909262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.909632 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.917977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.919557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:59.920752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 297/593] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:59.922005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:29:59.937559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:59.938959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:59.940020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:29:59.940020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:29:59.940020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:59.940020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:59.940020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:59.940020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:59.940020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:59.940020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:29:59.946172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 297/594] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:59.947146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.973443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:00.048672 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857cadf-1d86-4055-9575-632f47fa65ce None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.067517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c91d18-c165-4221-9d24-dd4b65b643ab None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.070215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:00.094534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.095121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.095399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.095672 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.096648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:00.096956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.099155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.111281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.113006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.113459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.113929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.122147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 298/595] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:00.123732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:00.127667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.130089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:00.151002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:00.152042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:00.152916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:00.152916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:00.152916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:00.157773 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 298/596] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:00.158933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:00.260896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:00.264924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc35dd-90ad-44d4-82a8-df28e2a3a1a9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.289093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ddb45-7bf5-4c10-8e77-d28f6bdbe687 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.302904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.303689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.304029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.304416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.305711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:00.306428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.307969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.326777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 299/597] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:00.327693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:00.330227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.331100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.331576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.332102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.344619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.346589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.359012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:00.363048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:00.364012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:00.364897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:00.364897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:00.364897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:00.367810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 299/598] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:00.378418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:00.445156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb1821-ebbe-4171-a4d5-87baa69353d4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.480647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19225a93-417b-4a59-93a6-7bef3c959e66 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.483245 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.483866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.484147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.484447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.485430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:00.485621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.486989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.505910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 300/599] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:00.507199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:00.512219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:00.522434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd513d-9294-42a2-9426-140ee90f3ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.528531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd513d-9294-42a2-9426-140ee90f3ae9 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.528531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd513d-9294-42a2-9426-140ee90f3ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.528531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd513d-9294-42a2-9426-140ee90f3ae9 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/b4d874536170494496769b4f37234b2f/roles/39294eb951d24b04be9b4b5d98b36640` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.541695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd513d-9294-42a2-9426-140ee90f3ae9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=b4d874536170494496769b4f37234b2f, role_id=39294eb951d24b04be9b4b5d98b36640)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.543728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd513d-9294-42a2-9426-140ee90f3ae9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.606376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:00.621655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42f24e-d4e7-4f94-aaff-3192c09287f5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.655105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.655753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.655999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.656245 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.656889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:00.657194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.658596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.675158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 301/600] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:00.675158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:00.775689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54bb2d-3b67-4a81-a8d0-87fda5306c60 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.809385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.810042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.810327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.810606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.817423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.819086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:00.830336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:00.831320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 302/601] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:00.832540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:00.932759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37978b1-2ddd-4042-a3b0-a64db7a7768d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.966763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.967385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.967666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.967936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.974540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.976620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.993885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 300/602] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:00 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/b4d874536170494496769b4f37234b2f/roles/39294eb951d24b04be9b4b5d98b36640 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:00.993885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae33dec-c3d2-45cb-9375-a4be1c2aa36f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.993885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae33dec-c3d2-45cb-9375-a4be1c2aa36f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:00.998182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:00.999021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:00.999951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:00.999951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:00.999951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:00.999951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:00.999951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:00.999951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:00.999951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:00.999951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:01.000403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae33dec-c3d2-45cb-9375-a4be1c2aa36f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.000488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 303/603] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:01.000488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:01.000670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 301/604] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:01.000670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae33dec-c3d2-45cb-9375-a4be1c2aa36f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:01.042002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:01.125524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798d722-5822-4942-80f0-7feceb672f15 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:01.139196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:01.152403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae33dec-c3d2-45cb-9375-a4be1c2aa36f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:01.167916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588c97a-13e9-41b5-915f-99f3a4136cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:01.169633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c97a-13e9-41b5-915f-99f3a4136cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.169968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c97a-13e9-41b5-915f-99f3a4136cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.170247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c97a-13e9-41b5-915f-99f3a4136cd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.170955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588c97a-13e9-41b5-915f-99f3a4136cd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:01.171262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588c97a-13e9-41b5-915f-99f3a4136cd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:01.172708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588c97a-13e9-41b5-915f-99f3a4136cd1 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:01.191221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 304/605] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:01.192301 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf66cf-9164-4bba-a4d7-bc69bd08f25a None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.192589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf66cf-9164-4bba-a4d7-bc69bd08f25a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.192863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf66cf-9164-4bba-a4d7-bc69bd08f25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.201455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:01.202484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.202996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.203479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.204471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:01.204984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:01.207244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:01.212506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-cfbf66cf-9164-4bba-a4d7-bc69bd08f25a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:01.213923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cfbf66cf-9164-4bba-a4d7-bc69bd08f25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:01.222623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 302/606] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:01.223613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:01.263815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:01.286237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 305/607] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:01.286982 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf66cf-9164-4bba-a4d7-bc69bd08f25a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:01.347473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95401d-c07f-4b89-a68c-1a9807db4739 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:01.374239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:01.392118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf66cf-9164-4bba-a4d7-bc69bd08f25a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:01.394373 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ce417-6b93-4ad9-8e52-791677be2260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:01.396029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ce417-6b93-4ad9-8e52-791677be2260 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.396546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ce417-6b93-4ad9-8e52-791677be2260 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.397137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ce417-6b93-4ad9-8e52-791677be2260 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/6c7a064cf91243648c6cc6d6ca576505/roles/39294eb951d24b04be9b4b5d98b36640` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.414718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ce417-6b93-4ad9-8e52-791677be2260 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=6c7a064cf91243648c6cc6d6ca576505, role_id=39294eb951d24b04be9b4b5d98b36640)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:01.416161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ce417-6b93-4ad9-8e52-791677be2260 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:01.438154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1302ec7-2e65-4406-bfd9-1d4b6d46c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:01.439322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1302ec7-2e65-4406-bfd9-1d4b6d46c5af None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.439790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1302ec7-2e65-4406-bfd9-1d4b6d46c5af None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.440223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1302ec7-2e65-4406-bfd9-1d4b6d46c5af None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/efc068011c14492d9ba483650cf632e5/roles/39294eb951d24b04be9b4b5d98b36640` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.456994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1302ec7-2e65-4406-bfd9-1d4b6d46c5af None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=efc068011c14492d9ba483650cf632e5, role_id=39294eb951d24b04be9b4b5d98b36640)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:01.458753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1302ec7-2e65-4406-bfd9-1d4b6d46c5af None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:01.502862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 303/608] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:01 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/6c7a064cf91243648c6cc6d6ca576505/roles/39294eb951d24b04be9b4b5d98b36640 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:01.505307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88afdc-f6ae-4eb2-8d96-efc1b4b3f413 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.506459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88afdc-f6ae-4eb2-8d96-efc1b4b3f413 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.506877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88afdc-f6ae-4eb2-8d96-efc1b4b3f413 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.512709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 304/609] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:01.516087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eb363-708b-4378-9dbe-dd18d191c206 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.516087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eb363-708b-4378-9dbe-dd18d191c206 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.516087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eb363-708b-4378-9dbe-dd18d191c206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.541913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 306/610] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:01 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/efc068011c14492d9ba483650cf632e5/roles/39294eb951d24b04be9b4b5d98b36640 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:01.545765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a55eb363-708b-4378-9dbe-dd18d191c206 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:01.547113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a55eb363-708b-4378-9dbe-dd18d191c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:01.565724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1302ec7-2e65-4406-bfd9-1d4b6d46c5af None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:01.685885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 305/611] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:01.734795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1302ec7-2e65-4406-bfd9-1d4b6d46c5af None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:01.744172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:01.777061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8648d-9138-4938-b5a5-a731dc084c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:01.777693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8648d-9138-4938-b5a5-a731dc084c86 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.778006 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8648d-9138-4938-b5a5-a731dc084c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.778280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8648d-9138-4938-b5a5-a731dc084c86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.790728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8648d-9138-4938-b5a5-a731dc084c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-a2d8648d-9138-4938-b5a5-a731dc084c86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:01.795438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:01.802101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:01.802996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 307/612] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:01.807003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eb363-708b-4378-9dbe-dd18d191c206 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:01.841662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:01.974593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eb363-708b-4378-9dbe-dd18d191c206 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.007057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8648d-9138-4938-b5a5-a731dc084c86 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.022606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.025693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.025693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.025693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.025693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:02.025693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.026289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.035356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 306/613] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:02.041255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.150218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:02.183553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8648d-9138-4938-b5a5-a731dc084c86 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.220040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c004b-3614-474e-8336-52817ae6ba35 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.220992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1147de-7228-485c-bc3d-ef69be2f6b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.221795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1147de-7228-485c-bc3d-ef69be2f6b68 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.222259 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1147de-7228-485c-bc3d-ef69be2f6b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.222679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1147de-7228-485c-bc3d-ef69be2f6b68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.234741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1147de-7228-485c-bc3d-ef69be2f6b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-0f1147de-7228-485c-bc3d-ef69be2f6b68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:02.240404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:02.242252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:02.243284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:02.245754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 308/614] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:02.254505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1147de-7228-485c-bc3d-ef69be2f6b68 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.299005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.299656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.299934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.300204 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.314494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:02.322463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.324745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:02.357647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:02.358875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:02.359976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:02.359976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:02.359976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:02.360198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 307/615] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:02.367808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.480194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cbf29-2c37-4b46-92f3-df273300cafd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.496133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1147de-7228-485c-bc3d-ef69be2f6b68 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.513267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:02.516936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.517753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.518153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.518521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.519295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:02.519693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.521140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.534812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 308/616] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:02.541525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.579826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.580586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.580973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.581348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.582167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:02.582575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.584548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.609991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 309/617] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:02.626049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.747755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586b27a-2669-412a-a2dd-e89528bca2c3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.801990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:02.807801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.808569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.808956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.809334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.817486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.820187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:02.832633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:02.833802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:02.835635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:02.836530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 309/618] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:02.843060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.845290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276635f4-338f-418a-8296-6cc7622808c0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.930768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.931540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.931915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.932283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.948898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:02.953793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.955388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:02.989863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:02.991060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:02.992153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:02.992153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:02.992153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:02.998252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 310/619] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:03.000675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:03.057096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b97b7e-46b0-4511-a09d-4773787ef550 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:03.129208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:03.130008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.130699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.133987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.134852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:03.135306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:03.136860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:03.177938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 310/620] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:03.178910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:03.195802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c5ff4-1907-4c85-bd29-4ed16089c0ce None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:03.237680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d4986-c1cd-4b64-aa41-4eeff06f3820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:03.238715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d4986-c1cd-4b64-aa41-4eeff06f3820 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.239209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d4986-c1cd-4b64-aa41-4eeff06f3820 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.239703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d4986-c1cd-4b64-aa41-4eeff06f3820 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.240591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d4986-c1cd-4b64-aa41-4eeff06f3820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:03.241124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d4986-c1cd-4b64-aa41-4eeff06f3820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:03.243014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d4986-c1cd-4b64-aa41-4eeff06f3820 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:03.252928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:03.258491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 311/621] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:03.260291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eedd10-92dc-48aa-bc90-d32fd662e713 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.260788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eedd10-92dc-48aa-bc90-d32fd662e713 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.261273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eedd10-92dc-48aa-bc90-d32fd662e713 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.263469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 312/622] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:03.264409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eedd10-92dc-48aa-bc90-d32fd662e713 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:03.346925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba12fd1-7b80-44cb-8154-9c7dff3aca41 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:03.360842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:03.385718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eedd10-92dc-48aa-bc90-d32fd662e713 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:03.391366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b23d-4c02-48eb-8a9e-16dec42b7a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:03.391989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b23d-4c02-48eb-8a9e-16dec42b7a4d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.392286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b23d-4c02-48eb-8a9e-16dec42b7a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.392562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b23d-4c02-48eb-8a9e-16dec42b7a4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.428207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f0343-a869-4d39-b58e-5bc4252d5ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:03.429273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f0343-a869-4d39-b58e-5bc4252d5ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.429825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f0343-a869-4d39-b58e-5bc4252d5ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.430384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f0343-a869-4d39-b58e-5bc4252d5ca0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.439236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f0343-a869-4d39-b58e-5bc4252d5ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:03.441167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f0343-a869-4d39-b58e-5bc4252d5ca0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-8f9f0343-a869-4d39-b58e-5bc4252d5ca0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:03.454656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:03.455665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:03.456629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:03.460426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 313/623] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:03.462062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda95061-42d5-481c-a57d-eec973c9c540 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.462558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda95061-42d5-481c-a57d-eec973c9c540 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.463064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda95061-42d5-481c-a57d-eec973c9c540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.470881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b23d-4c02-48eb-8a9e-16dec42b7a4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:03.472369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b23d-4c02-48eb-8a9e-16dec42b7a4d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:03.484175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-eda95061-42d5-481c-a57d-eec973c9c540 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:03.485910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-eda95061-42d5-481c-a57d-eec973c9c540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:03.553115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:03.566852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 314/624] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:03.568145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda95061-42d5-481c-a57d-eec973c9c540 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:03.590693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 311/625] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:30:03.596584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b23d-4c02-48eb-8a9e-16dec42b7a4d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:03.616720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:03.691571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda95061-42d5-481c-a57d-eec973c9c540 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:03.732061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61812d05-e8f4-41e6-82d4-4b3fb33321b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:03.733061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812d05-e8f4-41e6-82d4-4b3fb33321b4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.733552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812d05-e8f4-41e6-82d4-4b3fb33321b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.734081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812d05-e8f4-41e6-82d4-4b3fb33321b4 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.735026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61812d05-e8f4-41e6-82d4-4b3fb33321b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:03.735670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61812d05-e8f4-41e6-82d4-4b3fb33321b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:03.737312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61812d05-e8f4-41e6-82d4-4b3fb33321b4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:03.760136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 315/626] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 17:30:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:03.761807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0216d43-5d9f-498c-a29f-dd0c26ccd58e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.762395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0216d43-5d9f-498c-a29f-dd0c26ccd58e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.762525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b23d-4c02-48eb-8a9e-16dec42b7a4d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:03.763046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0216d43-5d9f-498c-a29f-dd0c26ccd58e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.765218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 316/627] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:03.766288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0216d43-5d9f-498c-a29f-dd0c26ccd58e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:03.799220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c550-3428-46a0-ab43-04b5d7f53820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:03.799655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c550-3428-46a0-ab43-04b5d7f53820 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.799940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c550-3428-46a0-ab43-04b5d7f53820 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.800219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c550-3428-46a0-ab43-04b5d7f53820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.810855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466c550-3428-46a0-ab43-04b5d7f53820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:03.812242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466c550-3428-46a0-ab43-04b5d7f53820 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:03.826991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-4466c550-3428-46a0-ab43-04b5d7f53820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:03.828391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:03.829311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:03.830235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:03.830235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:03.830235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:03.830235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 312/628] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:03.831768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed670e3-850f-477a-aacd-c384b18764bf None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.831768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed670e3-850f-477a-aacd-c384b18764bf None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.831768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed670e3-850f-477a-aacd-c384b18764bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.855353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-aed670e3-850f-477a-aacd-c384b18764bf None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:03.856710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-aed670e3-850f-477a-aacd-c384b18764bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:03.892269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0216d43-5d9f-498c-a29f-dd0c26ccd58e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:03.923529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:03.933638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f229c11-ee04-415a-9143-4ce46b5ded9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:03.934315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229c11-ee04-415a-9143-4ce46b5ded9a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.934582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229c11-ee04-415a-9143-4ce46b5ded9a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.934856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229c11-ee04-415a-9143-4ce46b5ded9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.936706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 313/629] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:03.937766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed670e3-850f-477a-aacd-c384b18764bf None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:03.953736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f229c11-ee04-415a-9143-4ce46b5ded9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:03.956191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f229c11-ee04-415a-9143-4ce46b5ded9a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-1f229c11-ee04-415a-9143-4ce46b5ded9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:03.974225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:03.975300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:03.976284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:03.976284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:03.976284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:03.978002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 317/630] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:03.979493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f281-c9c6-4305-99f2-8b01c49ef28a None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:03.979875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f281-c9c6-4305-99f2-8b01c49ef28a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:03.980253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f281-c9c6-4305-99f2-8b01c49ef28a None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.981674 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 318/631] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:03.982990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87f281-c9c6-4305-99f2-8b01c49ef28a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.048911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed670e3-850f-477a-aacd-c384b18764bf None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.087752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.088374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.088551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.088822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.089474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:04.089778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:04.091160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:04.095901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:04.099935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87f281-c9c6-4305-99f2-8b01c49ef28a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.107961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 314/632] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:04.107961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.136396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c994ed-a432-4fcd-994f-ee287d54ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.137022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c994ed-a432-4fcd-994f-ee287d54ae72 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.137303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c994ed-a432-4fcd-994f-ee287d54ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.137578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c994ed-a432-4fcd-994f-ee287d54ae72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.148485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c994ed-a432-4fcd-994f-ee287d54ae72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-27c994ed-a432-4fcd-994f-ee287d54ae72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:04.153564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:04.154665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:04.155518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 319/633] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:04.156245 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf300-fe20-473d-808c-1e204580c54b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.156523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf300-fe20-473d-808c-1e204580c54b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.156795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf300-fe20-473d-808c-1e204580c54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.167368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:04.178797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-c9dcf300-fe20-473d-808c-1e204580c54b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:04.180479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c9dcf300-fe20-473d-808c-1e204580c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:04.221831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c3635-1c51-4676-b693-f2574399267a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.251194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 320/634] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:04.252061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcf300-fe20-473d-808c-1e204580c54b None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.257295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.257922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.258204 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.258475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.259115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:04.259415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:04.260790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:04.273621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 315/635] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:04.274610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.358229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:04.369823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcf300-fe20-473d-808c-1e204580c54b None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.384598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2521734-fbf8-44db-8692-566e76c31393 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.395711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:04.409015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0953195-326c-4481-8892-f620980df885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.409641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953195-326c-4481-8892-f620980df885 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.409948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953195-326c-4481-8892-f620980df885 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.410219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953195-326c-4481-8892-f620980df885 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.417836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0953195-326c-4481-8892-f620980df885 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:04.419354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0953195-326c-4481-8892-f620980df885 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:04.420210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.420891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.421181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.421484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.422160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:04.422467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:04.423827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c0953195-326c-4481-8892-f620980df885 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:04.436695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:04.437657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:04.438900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:04.438900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:04.438900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:04.438900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:04.438900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:04.438900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:04.438900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:04.438900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:04.439437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 316/636] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:04.439437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.439597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 321/637] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:04.439597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0953195-326c-4481-8892-f620980df885 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.542671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17cc32-b117-457c-bc9b-98d6eac05093 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.545932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0953195-326c-4481-8892-f620980df885 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.578000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55191ec3-1133-4b7d-92a8-53f65a2e21cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.578634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191ec3-1133-4b7d-92a8-53f65a2e21cd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.578918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191ec3-1133-4b7d-92a8-53f65a2e21cd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.579189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191ec3-1133-4b7d-92a8-53f65a2e21cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.582142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.582773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.583053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.583325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.590803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:04.590953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55191ec3-1133-4b7d-92a8-53f65a2e21cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:04.592243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-55191ec3-1133-4b7d-92a8-53f65a2e21cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:04.596055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:04.597095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:04.598250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 317/638] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:04.598947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55191ec3-1133-4b7d-92a8-53f65a2e21cd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:04.603746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:04.604766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:04.605668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 322/639] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:04.606395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.620089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:04.633483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:04.701522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55191ec3-1133-4b7d-92a8-53f65a2e21cd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.709547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5263d1-3097-4d67-9c9e-1e9e6cc44a73 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.734111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.734734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.735015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.735288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.744792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.745416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.745870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.746152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.746479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:04.746884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:04.747157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:04.747922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:04.748561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:04.761638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 323/640] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:04.762615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:04.762744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:04.763876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:04.765011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:04.765011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:04.765011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:04.766157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 318/641] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:04.767172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.835278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:04.851714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:04.866577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3527c-f57b-4115-ad60-7d074494036d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.867406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e869e-7c2d-4f81-bdcb-c8a0eb5e2580 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:04.900829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.901348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.901658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.901767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aae975-d548-40af-9598-7a1f79edbe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:04.901976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.902521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aae975-d548-40af-9598-7a1f79edbe08 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:04.902644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:04.902904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aae975-d548-40af-9598-7a1f79edbe08 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:04.902996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:04.903281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aae975-d548-40af-9598-7a1f79edbe08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:04.904295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:04.913964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 319/642] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:04.914790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:04.922065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aae975-d548-40af-9598-7a1f79edbe08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-b5aae975-d548-40af-9598-7a1f79edbe08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:04.926757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:04.927893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:04.929131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 324/643] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:04.929970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aae975-d548-40af-9598-7a1f79edbe08 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.021277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573af80-f3d0-4b1d-bf81-84ec6164163e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.031604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aae975-d548-40af-9598-7a1f79edbe08 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.057394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.058022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.058335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.058685 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.059462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:05.059757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.061163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.061286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:05.067605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.068249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.069015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.069285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.070003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:05.070309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.071685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.073821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 320/644] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:05.074639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.083213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:05.085107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 325/645] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:05.087024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.175130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cd7eb-c257-4ba1-9500-663fe2a1dea4 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.186566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd78e66-02d7-42a0-9de2-aa13948cec1c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.208664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.209824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.210134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.210398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.211119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=20dd03dd2fa04147bba827f983a58034) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:05.211424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.212841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.218594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 321/646] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/role_assignments?user.id=20dd03dd2fa04147bba827f983a58034&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:05.219384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.222744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.223512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.223913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.224283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.236580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.238243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:05.256354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:05.257597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:05.258763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:05.258763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:05.258763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:05.264538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 326/647] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:05.264538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.305532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:05.316403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:05.342025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d0ac-f04e-48f4-a44c-03d34203afb2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.382266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.383144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.383144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.383412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.384083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:05.384401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.385745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.391848 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4f3d9-bd7a-4843-ad7e-27171b13e6b2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.395470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 322/648] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:05.396783 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.436257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2c55e-7718-41e2-bec6-6b001c260204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.437114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2c55e-7718-41e2-bec6-6b001c260204 None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.437512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2c55e-7718-41e2-bec6-6b001c260204 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.437800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2c55e-7718-41e2-bec6-6b001c260204 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/e698e658de56495e8d501ca6dfd75623/roles/39294eb951d24b04be9b4b5d98b36640` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.454073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2c55e-7718-41e2-bec6-6b001c260204 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=e698e658de56495e8d501ca6dfd75623, role_id=39294eb951d24b04be9b4b5d98b36640)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.455740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2c55e-7718-41e2-bec6-6b001c260204 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.504991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fac43-ca7d-4a1a-b088-2a70f472daab None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.535691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:05.541014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.541648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.541968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.542250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.549780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.551253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:05.562362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:05.563372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 323/650] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:05.569703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.574354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 327/650] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:05 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/e698e658de56495e8d501ca6dfd75623/roles/39294eb951d24b04be9b4b5d98b36640 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:05.575398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2c55e-7718-41e2-bec6-6b001c260204 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.694005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:05.703832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2c55e-7718-41e2-bec6-6b001c260204 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.726047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22a1e4-c6b8-41e3-ba87-6d641431511e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.754159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.755153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.755720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.756778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.765836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.766472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.766757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.767038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.767699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:05.768012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.769385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.769680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.772232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.782540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 324/651] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:05.783484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:05.796315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:05.797516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:05.798717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:05.798717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:05.798717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:05.801940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 328/652] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:05.803810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.854080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:05.912304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa99e6-05e5-40b7-a2bf-cd40dc7aedda None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.942224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ed24e-fe96-4ec4-9c23-947e52ae4e0d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:05.962475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.963383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.963760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.964132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.964969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:05.965424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:05.967124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:05.986693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:05.987510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 325/653] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:05.987635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:05.987874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:05.988148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:05.988714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:05.998963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:06.000420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:06.016686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:06.017900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:06.019398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 329/654] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:06.024695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:06.029993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:06.109429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac42b-0e14-4f3b-87fc-1d837a3ef381 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:06.122306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:06.152393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d0177-ef8e-4b75-b3f5-5ddcc05f972e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:06.157389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:06.158247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:06.158671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:06.159084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:06.159867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:06.160734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:06.163183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:06.184391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 326/655] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:06.184391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:06.197018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bed59-1aab-4fd1-87ac-3446fde4eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:06.198331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bed59-1aab-4fd1-87ac-3446fde4eecd None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:06.198781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bed59-1aab-4fd1-87ac-3446fde4eecd None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:06.199222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bed59-1aab-4fd1-87ac-3446fde4eecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:06.279977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bed59-1aab-4fd1-87ac-3446fde4eecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:06.281899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bed59-1aab-4fd1-87ac-3446fde4eecd None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:06.294232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:06.295710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb105a-0564-4ede-a5ec-825129001947 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:06.331089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:06.331956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:06.332334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:06.332751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:06.333604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:06.334594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:06.336172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:06.356048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 327/656] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 17:30:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:06.357688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:06.386740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:06.405481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 330/657] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:30:06.406730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bed59-1aab-4fd1-87ac-3446fde4eecd None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:06.464610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8104c-4ada-4ff0-98ab-39059053252d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:06.501017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:06.501794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:06.502236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:06.502614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:06.511278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:06.512825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:06.522422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bed59-1aab-4fd1-87ac-3446fde4eecd None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:06.525165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:06.526703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:06.527810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:30:06.528931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:06.541667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 328/658] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:06.541667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:06.573239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c82ded-18a0-43c1-b0c4-efb9c53d46b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:06.574347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c82ded-18a0-43c1-b0c4-efb9c53d46b8 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:06.575175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c82ded-18a0-43c1-b0c4-efb9c53d46b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:06.575713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c82ded-18a0-43c1-b0c4-efb9c53d46b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:06.641948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:06.654243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c82ded-18a0-43c1-b0c4-efb9c53d46b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:06.654583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85728fb-09d3-4d00-afed-98fb9ac2db62 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:06.656222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c82ded-18a0-43c1-b0c4-efb9c53d46b8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:06.692110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:06.692890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:06.693407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:06.693789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:06.744813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:06.744813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:06.768307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:06.769469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:06.770545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:06.770545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:06.770545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:06.781883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 329/659] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:06.786122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:06.877609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 331/660] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:30:06.878652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c82ded-18a0-43c1-b0c4-efb9c53d46b8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:06.887237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:06.988044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08433b-f5c0-48e9-a57d-9f7ee73eb71d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.031072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.033642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.034105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.034567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.035491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:07.036830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.038500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.055875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:07.067958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 330/661] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 24 17:30:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:07.069768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:07.082484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c82ded-18a0-43c1-b0c4-efb9c53d46b8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.156691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.157416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.157791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.158222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.158998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:07.159409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.160898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.186211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 332/662] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:07.186211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:07.264026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e1e91-a6eb-4c93-8db6-b35970a880a8 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.280529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:07.309479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.310313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.310718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.311210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.326926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.329165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.332681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffc125-79a7-4d6e-837c-73947d1ccbc8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:07.349124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:07.350734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:07.351802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:07.351802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:07.351802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:07.351802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 331/663] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:07.361878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:07.376681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f3ab2-a30f-4fdb-8cbf-c36ae1c2a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.377293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3ab2-a30f-4fdb-8cbf-c36ae1c2a522 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.377580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3ab2-a30f-4fdb-8cbf-c36ae1c2a522 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.377876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3ab2-a30f-4fdb-8cbf-c36ae1c2a522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.406647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:07.457331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f3ab2-a30f-4fdb-8cbf-c36ae1c2a522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.458287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac207b-0deb-4b42-8d06-c7e3a709849f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.458894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f3ab2-a30f-4fdb-8cbf-c36ae1c2a522 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.494676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1017c04-e6d9-444d-b055-989ffb42d8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.495344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1017c04-e6d9-444d-b055-989ffb42d8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.495628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1017c04-e6d9-444d-b055-989ffb42d8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.495907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1017c04-e6d9-444d-b055-989ffb42d8a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.502752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1017c04-e6d9-444d-b055-989ffb42d8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.504205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1017c04-e6d9-444d-b055-989ffb42d8a4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c1017c04-e6d9-444d-b055-989ffb42d8a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:07.516889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:07.517904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:07.518805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 332/664] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:07.519301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545161c-2efc-4106-94f0-6164e15d2445 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.521033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545161c-2efc-4106-94f0-6164e15d2445 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.521033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545161c-2efc-4106-94f0-6164e15d2445 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.521933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 333/665] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:07.522681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545161c-2efc-4106-94f0-6164e15d2445 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:07.532647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:07.574188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 333/666] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:30:07.575387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f3ab2-a30f-4fdb-8cbf-c36ae1c2a522 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:07.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545161c-2efc-4106-94f0-6164e15d2445 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.664682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:07.679326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f3ab2-a30f-4fdb-8cbf-c36ae1c2a522 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.689012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b614d-46ed-45fd-9b3f-5503569ed33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.689637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b614d-46ed-45fd-9b3f-5503569ed33f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.689945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b614d-46ed-45fd-9b3f-5503569ed33f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.690290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b614d-46ed-45fd-9b3f-5503569ed33f None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.690936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b614d-46ed-45fd-9b3f-5503569ed33f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:07.691242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b614d-46ed-45fd-9b3f-5503569ed33f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.692607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b614d-46ed-45fd-9b3f-5503569ed33f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.706456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 334/667] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:07.707566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9152c-af27-4d90-824f-2f44ae536693 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.707972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9152c-af27-4d90-824f-2f44ae536693 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.708607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9152c-af27-4d90-824f-2f44ae536693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.718356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.718999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.719281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.719558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.720266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:07.720564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.722059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.726421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-49b9152c-af27-4d90-824f-2f44ae536693 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:07.727676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-49b9152c-af27-4d90-824f-2f44ae536693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:07.733935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 334/668] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:07.734970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:07.761244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:07.801759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 335/669] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:07.803707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df8d8c-4947-4ca2-8baa-a78f48090f59 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.803990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df8d8c-4947-4ca2-8baa-a78f48090f59 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.804278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df8d8c-4947-4ca2-8baa-a78f48090f59 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.809866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 336/670] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:07.809866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df8d8c-4947-4ca2-8baa-a78f48090f59 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:07.836494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58681eb-e3d7-447b-8684-978afec6601f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.872014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.872654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.872938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.873214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.885376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.886858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:07.901574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:07.902593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:07.903526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:07.903526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:07.903526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:07.907410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 335/671] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:07.908287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:07.918880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:07.923760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df8d8c-4947-4ca2-8baa-a78f48090f59 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:07.965154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced947a-a3cb-41f0-bb77-0ace987715de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:07.965691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced947a-a3cb-41f0-bb77-0ace987715de None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.966013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced947a-a3cb-41f0-bb77-0ace987715de None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.966291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced947a-a3cb-41f0-bb77-0ace987715de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:07.973143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced947a-a3cb-41f0-bb77-0ace987715de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:07.975199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced947a-a3cb-41f0-bb77-0ace987715de None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-bced947a-a3cb-41f0-bb77-0ace987715de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:07.986410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:07.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:07.988172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 337/672] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:07.989011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920c8bf-2b3e-468b-a6e4-653553e53b79 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:07.989011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920c8bf-2b3e-468b-a6e4-653553e53b79 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:07.989241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920c8bf-2b3e-468b-a6e4-653553e53b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.005286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:08.008466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4920c8bf-2b3e-468b-a6e4-653553e53b79 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:08.009680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4920c8bf-2b3e-468b-a6e4-653553e53b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:08.032482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad356ee7-fad3-41ce-896c-d95215b57bdb None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.070722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.071346 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.071700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.071965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.072690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=b4d874536170494496769b4f37234b2f) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:08.073050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:08.074531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:08.080516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 336/673] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:07 2022] GET /identity/v3/role_assignments?user.id=b4d874536170494496769b4f37234b2f&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:08.081316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.082077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 338/674] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:08.083076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920c8bf-2b3e-468b-a6e4-653553e53b79 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.207509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2563b-c153-4c18-9c7c-028ef6c48166 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.210228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:08.225591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920c8bf-2b3e-468b-a6e4-653553e53b79 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.252454 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.253137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.253423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.253721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.254485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:08.255109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:08.256715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:08.267568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:08.269160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.269787 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.270089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.270368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.271074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:08.271928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:08.272776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 337/675] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:08.273376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:08.273716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.286283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 339/676] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:08.287164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.408195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568191-b1d5-434f-9b29-27b598f55e72 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.415140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f81a6-365c-44e5-a2bc-8fcd8354eb4b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.447870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edd5d3-f670-4f2f-b00c-b1d439803b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.448503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edd5d3-f670-4f2f-b00c-b1d439803b13 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.448830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edd5d3-f670-4f2f-b00c-b1d439803b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.449105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edd5d3-f670-4f2f-b00c-b1d439803b13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.455318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.455946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.456231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.456507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.461413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edd5d3-f670-4f2f-b00c-b1d439803b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:08.463868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:08.465318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-87edd5d3-f670-4f2f-b00c-b1d439803b13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:08.467110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:08.468010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:08.468862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:08.470704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 338/677] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:08.471396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edd5d3-f670-4f2f-b00c-b1d439803b13 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:08.484032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:08.484967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:08.485883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 340/678] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:08.486381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.502999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:08.544517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:08.589699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edd5d3-f670-4f2f-b00c-b1d439803b13 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.602860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a564f-0119-43f4-894a-56e6afc7853c None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.627455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.628056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.628338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.628606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.629449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=6c7a064cf91243648c6cc6d6ca576505) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:08.629765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:08.631995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:08.641589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 339/679] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/role_assignments?user.id=6c7a064cf91243648c6cc6d6ca576505&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:08.642634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.646906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f42617-100a-4931-a883-51d392880734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.647780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f42617-100a-4931-a883-51d392880734 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.648139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f42617-100a-4931-a883-51d392880734 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.648534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f42617-100a-4931-a883-51d392880734 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.664540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f42617-100a-4931-a883-51d392880734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-d6f42617-100a-4931-a883-51d392880734 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:08.670820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:08.672019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:08.673052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:08.681473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 341/680] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:08.682771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f42617-100a-4931-a883-51d392880734 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.774968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:08.781231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585443c-8e4e-4b69-b0a5-eea4d6146cf4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.809934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:08.813890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f42617-100a-4931-a883-51d392880734 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:08.833319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.834751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.834937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.835759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.836770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:08.837083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:08.838486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:08.852270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 340/681] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:08.853430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.873603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:08.874279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:08.874559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:08.874833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:08.875542 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:08.875846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:08.877381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:08.893938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 342/682] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:08.895349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.984584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5ed5b-dc39-46a2-8cdf-ffe3e22472cb None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.022862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.023602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.023891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.024172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.027948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:09.027948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.027948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.036251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 341/683] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:09.037220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:09.048322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c2cb-8846-4c99-ace0-2d2010a1763b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.083628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:09.089254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.090122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.090478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.090819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.102314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.103862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.107224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:09.120068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:09.121231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:09.122344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:09.122344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:09.122344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:09.128175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 343/684] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:09.128175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:09.172540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26833ff-cf2c-407a-8c84-1f85a81c85db None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.219542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.220175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.220413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.220686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.221415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=efc068011c14492d9ba483650cf632e5) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:09.221724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.223737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.238181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 342/685] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/role_assignments?user.id=efc068011c14492d9ba483650cf632e5&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:09.239221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:09.261483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86064007-4322-49ce-b79c-7933f9e21a33 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.307963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.309328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.310168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.310594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.327286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.328992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:09.347608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:09.348811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:09.350714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:09.350714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:09.350714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:09.352790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 344/686] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:09.359005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:09.405408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:09.483646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351fc09-ccb7-4d07-818c-c445b657cbd2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.501704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff805-1bf5-491a-9715-77039fba49bb None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.517740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:09.541620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.542396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.542587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.542866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.543515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:09.544044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.545513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.560465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 345/687] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:09.561536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.562416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.562921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.563646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.564489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:09.565099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.566726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.570917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:09.581338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 343/688] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:09.589265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:09.696812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:09.704239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a78172-f4bb-4f8f-b97d-31ed6807408b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.745559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.746359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.746640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.746913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] PATH_INFO: `/v3/users/swift` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.755697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.757219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.762954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651a61-434a-4f63-816d-b68216526e6e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:09.770246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:09.771282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:09.772682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:09.774179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 346/689] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:09.778982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:09.870488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.871308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.871599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.871940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.898086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3cb3a-e7a1-4b05-a826-98bfb71351db None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.898689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.900233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:09.928696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:09.930895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:09.933119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:09.939234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 344/690] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:09.946694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:09.948843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.949461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.949739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.950043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.951121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:09.951432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.952855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.978515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 347/691] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:09.980247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:10.010258 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:10.012620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:10.090087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db623d81-9d25-4a4d-b5bf-084dc6b052ea None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:10.129072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7bf33-25ff-475e-9045-403ebbce0b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:10.129702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7bf33-25ff-475e-9045-403ebbce0b1a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.130020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7bf33-25ff-475e-9045-403ebbce0b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.130288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7bf33-25ff-475e-9045-403ebbce0b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.133334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a948e-0239-4874-ad8a-99c3d90470d4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:10.204540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7bf33-25ff-475e-9045-403ebbce0b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:10.206031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7bf33-25ff-475e-9045-403ebbce0b1a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:10.238233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:10.258632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8b3b0-4ce6-4ae3-b64c-4b7d80347e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:10.259299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8b3b0-4ce6-4ae3-b64c-4b7d80347e29 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.259531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8b3b0-4ce6-4ae3-b64c-4b7d80347e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.259801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8b3b0-4ce6-4ae3-b64c-4b7d80347e29 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.260541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8b3b0-4ce6-4ae3-b64c-4b7d80347e29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:10.260844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8b3b0-4ce6-4ae3-b64c-4b7d80347e29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:10.262495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8b3b0-4ce6-4ae3-b64c-4b7d80347e29 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:10.282360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 345/692] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:10.283769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e0dd0-5910-4c14-9740-42e680f20404 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.284053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e0dd0-5910-4c14-9740-42e680f20404 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.284326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e0dd0-5910-4c14-9740-42e680f20404 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.290178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 346/693] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:10.291013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e0dd0-5910-4c14-9740-42e680f20404 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:10.320096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 348/694] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:30:10.326776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12b977-bcfe-4066-94eb-e7641792b27f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.327165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12b977-bcfe-4066-94eb-e7641792b27f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.327445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12b977-bcfe-4066-94eb-e7641792b27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.346724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-dc12b977-bcfe-4066-94eb-e7641792b27f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:10.347916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-dc12b977-bcfe-4066-94eb-e7641792b27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:10.392375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:10.404305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e0dd0-5910-4c14-9740-42e680f20404 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:10.423224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 349/695] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:10.424550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12b977-bcfe-4066-94eb-e7641792b27f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:10.464964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f44374-2179-4567-ad2e-3edf3141137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:10.465584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44374-2179-4567-ad2e-3edf3141137a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.465879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44374-2179-4567-ad2e-3edf3141137a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.466168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44374-2179-4567-ad2e-3edf3141137a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.471812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:10.532457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12b977-bcfe-4066-94eb-e7641792b27f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:10.578069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:10.579080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.580480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.581531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.601470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:10.605705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:10.610568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f44374-2179-4567-ad2e-3edf3141137a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:10.612166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f44374-2179-4567-ad2e-3edf3141137a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:10.636103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:10.637295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:10.638432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:10.638432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:10.638432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:10.648559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 350/696] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:10.651214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:10.771707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:10.784937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 347/697] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:30:10.793195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5f91-e7c7-4773-a51f-38d1725b8e99 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.793195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5f91-e7c7-4773-a51f-38d1725b8e99 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.793195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5f91-e7c7-4773-a51f-38d1725b8e99 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.793195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 348/698] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:10.793195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5f91-e7c7-4773-a51f-38d1725b8e99 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:10.825947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:10.833048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4271a1b-464a-4758-b13a-651fcc392276 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:10.867653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06737ed3-532f-4f3b-8471-461be53a06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:10.868255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737ed3-532f-4f3b-8471-461be53a06a4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.868532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737ed3-532f-4f3b-8471-461be53a06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.868928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737ed3-532f-4f3b-8471-461be53a06a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.869574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06737ed3-532f-4f3b-8471-461be53a06a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:10.869920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06737ed3-532f-4f3b-8471-461be53a06a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:10.871287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06737ed3-532f-4f3b-8471-461be53a06a4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:10.890212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 351/699] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:10.891607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff13cbe-5cc9-4771-ac8d-c7e0f0f214f6 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.891892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff13cbe-5cc9-4771-ac8d-c7e0f0f214f6 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.892222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff13cbe-5cc9-4771-ac8d-c7e0f0f214f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.917683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-6ff13cbe-5cc9-4771-ac8d-c7e0f0f214f6 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:10.918837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6ff13cbe-5cc9-4771-ac8d-c7e0f0f214f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:10.919532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5f91-e7c7-4773-a51f-38d1725b8e99 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:10.960667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:10.961794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:10.961794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:10.961794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:10.972270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:10.973704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:10.989779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:10.991637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:10.992520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:10.992520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:10.992520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:10.992520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 349/700] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:10.992757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.002059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 352/701] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:11.002873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff13cbe-5cc9-4771-ac8d-c7e0f0f214f6 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.059689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:11.102740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:11.124617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff13cbe-5cc9-4771-ac8d-c7e0f0f214f6 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.124991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94e1ed-61dd-463a-b2f7-1bc7dc0620e7 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.161542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.162817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.163137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.163288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.163429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.164393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.164854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.165300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.172634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.174365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.175444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.176884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:11.187479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:11.188690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:11.189652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:11.189652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:11.189652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:11.189652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:11.189652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:11.189652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:11.189652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:11.189652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:11.197930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:11.198887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:11.199790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:11.199790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:11.199790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:11.200616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 353/702] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:11.201689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.202263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 350/703] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:11.203090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.306293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:11.311675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740a86-0942-4b33-94bb-66aab8e83924 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.312920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad04a4a-3fa9-462f-9bc4-415176b64566 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.350104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52105437-57e3-443a-b023-3d4218139aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.351377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105437-57e3-443a-b023-3d4218139aac None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.351945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105437-57e3-443a-b023-3d4218139aac None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.352400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105437-57e3-443a-b023-3d4218139aac None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.352506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.353006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.353286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.353616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.353876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52105437-57e3-443a-b023-3d4218139aac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:11.354212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52105437-57e3-443a-b023-3d4218139aac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.354318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:11.354589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.355746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52105437-57e3-443a-b023-3d4218139aac None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.355931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.377365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 354/704] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:11.377365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.377655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 351/705] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:11.377655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52105437-57e3-443a-b023-3d4218139aac None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.382438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:11.486667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52105437-57e3-443a-b023-3d4218139aac None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.493093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e122a9-9059-40f5-829e-2b6fabb32235 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.523257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.523903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.524246 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.524550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.525151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.526619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.532664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.543478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.543478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.543478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.543478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:11.543478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.543478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:11.543893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:11.544812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:11.545675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 352/706] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:11.547631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.560264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 355/707] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:11.560264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.571260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:11.629135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:11.669293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd30dd9-86af-4a3a-9b92-47c661eec844 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.671240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15044d7-fb97-4b96-8023-bea25a18dec9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.706886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.707496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.707773 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.708043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.709400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=e698e658de56495e8d501ca6dfd75623) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:11.709982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.711501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.715337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.716440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.716899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.717407 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.717601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 356/708] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/role_assignments?user.id=e698e658de56495e8d501ca6dfd75623&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:11.718412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.734570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.736191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:11.749452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:11.750487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:11.751395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:11.751395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:11.751395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:11.751395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:11.751395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:11.751395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:11.751395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:11.751395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:11.756103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 353/709] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:11.757464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.848801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:11.851496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e01b3-dcd7-4604-b2b0-b6947559ee95 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.875290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481f46e-1b3d-4b56-b436-a0d0723b2925 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.882512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:11.891334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.891947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.892229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.892496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.893134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:11.893431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.894917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.904895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 357/710] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:11.904895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.923388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50775bd-6943-40df-a1ef-6226795c506a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.924005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50775bd-6943-40df-a1ef-6226795c506a None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.924307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50775bd-6943-40df-a1ef-6226795c506a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.924556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50775bd-6943-40df-a1ef-6226795c506a None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.925202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50775bd-6943-40df-a1ef-6226795c506a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.927191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50775bd-6943-40df-a1ef-6226795c506a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:12.014913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d3cfe-f20e-4e11-8937-59d09e482a40 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:12.033904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 354/711] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:30:11 2022] POST /identity/v3/users => generated 254 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:12.033904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9e33-aed9-4be9-916d-cdcc011e7d68 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:12.033904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9e33-aed9-4be9-916d-cdcc011e7d68 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:12.033904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9e33-aed9-4be9-916d-cdcc011e7d68 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:12.035073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 355/712] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:12.036378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9e33-aed9-4be9-916d-cdcc011e7d68 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:12.058360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b6012-5cf5-4d05-86ec-8d341f7c0e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:12.059055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b6012-5cf5-4d05-86ec-8d341f7c0e3a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:12.059337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b6012-5cf5-4d05-86ec-8d341f7c0e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:12.059612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b6012-5cf5-4d05-86ec-8d341f7c0e3a None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:12.060340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b6012-5cf5-4d05-86ec-8d341f7c0e3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:12.060650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b6012-5cf5-4d05-86ec-8d341f7c0e3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:12.062213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b6012-5cf5-4d05-86ec-8d341f7c0e3a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:12.082024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 358/713] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:12.085389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd018dc-4912-4582-bba5-8a4012b7d573 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:12.086284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd018dc-4912-4582-bba5-8a4012b7d573 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:12.086571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd018dc-4912-4582-bba5-8a4012b7d573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:12.110654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-cdd018dc-4912-4582-bba5-8a4012b7d573 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:12.111945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cdd018dc-4912-4582-bba5-8a4012b7d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:12.190192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:12.209665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9e33-aed9-4be9-916d-cdcc011e7d68 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:12.225369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 359/714] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:12.226335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd018dc-4912-4582-bba5-8a4012b7d573 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:12.252083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:12.253053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:12.254270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:12.254815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:12.256330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:12.257009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:12.259016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:12.261069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:12.273979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 356/715] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 17:30:12 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:12.386212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd018dc-4912-4582-bba5-8a4012b7d573 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:12.435094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:12.435711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:12.435998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:12.436274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:12.437022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=20dd03dd2fa04147bba827f983a58034) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:12.437334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:12.439086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:12.446583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 360/716] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:12 2022] GET /identity/v3/role_assignments?user.id=20dd03dd2fa04147bba827f983a58034&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:12.620452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:12.754940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9da26-246c-450d-9d94-0000e037e92f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:12.834020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:12.834020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:12.834020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:12.834020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] PATH_INFO: `/v3/services/network` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:12.834450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:12.834450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:12.851607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:12.852710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:12.853600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 357/717] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:12 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:12.859031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:13.003099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:13.097355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd9eb9-db0f-44fe-94b1-dc66fc82c582 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:13.155585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617a494-56a0-4535-ac70-02b221dca967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:13.156213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a494-56a0-4535-ac70-02b221dca967 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:13.156548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a494-56a0-4535-ac70-02b221dca967 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:13.156833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a494-56a0-4535-ac70-02b221dca967 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:13.157484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617a494-56a0-4535-ac70-02b221dca967 None admin] RBAC: Adding query filter params (name=network) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:13.157789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617a494-56a0-4535-ac70-02b221dca967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:13.159217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617a494-56a0-4535-ac70-02b221dca967 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:13.165072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 361/718] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:13.173317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:13.320019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:13.384742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b947d-dcc6-4dcc-96b4-a862ef129bbf None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:13.446808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c9b3a-f45d-4b64-9d63-b2c6e555e8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:13.447349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c9b3a-f45d-4b64-9d63-b2c6e555e8ff None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:13.447630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c9b3a-f45d-4b64-9d63-b2c6e555e8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:13.448132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c9b3a-f45d-4b64-9d63-b2c6e555e8ff None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:13.448538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c9b3a-f45d-4b64-9d63-b2c6e555e8ff None admin] RBAC: Adding query filter params (type=network) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:13.448840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c9b3a-f45d-4b64-9d63-b2c6e555e8ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:13.450225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c9b3a-f45d-4b64-9d63-b2c6e555e8ff None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:13.458045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 358/719] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:14.205464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304423f-fad3-45b6-b820-9d15584f9818 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:14.206487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304423f-fad3-45b6-b820-9d15584f9818 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:14.207014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304423f-fad3-45b6-b820-9d15584f9818 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:14.209449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 362/720] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:14.215635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cfaf9-0281-45bd-9bcf-a9d6a757bfd8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:14.216124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cfaf9-0281-45bd-9bcf-a9d6a757bfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:14.216208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cfaf9-0281-45bd-9bcf-a9d6a757bfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:14.242788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-fa1cfaf9-0281-45bd-9bcf-a9d6a757bfd8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:14.243995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-fa1cfaf9-0281-45bd-9bcf-a9d6a757bfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:14.348482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 359/721] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:14.678120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304423f-fad3-45b6-b820-9d15584f9818 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:14.797583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129a7f-3591-48e5-bd6e-3f595a59c7ff None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:14.797975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129a7f-3591-48e5-bd6e-3f595a59c7ff None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:14.798159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129a7f-3591-48e5-bd6e-3f595a59c7ff None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:14.799814 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 360/722] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:14.807329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24803df6-72fc-4499-acc2-ab91034bf3e1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:14.807628 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24803df6-72fc-4499-acc2-ab91034bf3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:14.807886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24803df6-72fc-4499-acc2-ab91034bf3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:14.838443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-24803df6-72fc-4499-acc2-ab91034bf3e1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:14.840419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-24803df6-72fc-4499-acc2-ab91034bf3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:14.848643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304423f-fad3-45b6-b820-9d15584f9818 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:14.863400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:14.888862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6d75a-de92-4640-8cf7-1b23bd76c04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:14.889594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6d75a-de92-4640-8cf7-1b23bd76c04c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:14.889974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6d75a-de92-4640-8cf7-1b23bd76c04c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:14.890317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6d75a-de92-4640-8cf7-1b23bd76c04c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:14.902723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6d75a-de92-4640-8cf7-1b23bd76c04c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-22d6d75a-de92-4640-8cf7-1b23bd76c04c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:14.909198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:14.910429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:14.911326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:14.913696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 363/723] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:14.915227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22619efc-9ef9-437e-b5d5-19b7afc7fb0b None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:14.915838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:14.917626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22619efc-9ef9-437e-b5d5-19b7afc7fb0b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:14.918554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22619efc-9ef9-437e-b5d5-19b7afc7fb0b None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:14.924434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 364/724] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:14.924434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88b6e4-2ce1-441e-9f9f-3cf71aff45e7 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:14.924434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88b6e4-2ce1-441e-9f9f-3cf71aff45e7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:14.924434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88b6e4-2ce1-441e-9f9f-3cf71aff45e7 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:14.924434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 365/725] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:14.928909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88b6e4-2ce1-441e-9f9f-3cf71aff45e7 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:14.939715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 361/726] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:30:14.941808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ce8a0-ba90-4a24-ac99-dad6bec58618 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:14.942285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ce8a0-ba90-4a24-ac99-dad6bec58618 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:14.942724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ce8a0-ba90-4a24-ac99-dad6bec58618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:14.979960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-848ce8a0-ba90-4a24-ac99-dad6bec58618 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:14.982249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-848ce8a0-ba90-4a24-ac99-dad6bec58618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:15.049009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88b6e4-2ce1-441e-9f9f-3cf71aff45e7 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:15.076099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 362/727] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:15.077713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39886ac-790f-4c74-84c4-53b16662ce02 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:15.078126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39886ac-790f-4c74-84c4-53b16662ce02 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:15.078639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39886ac-790f-4c74-84c4-53b16662ce02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:15.086814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:15.087444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:15.087801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:15.088080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:15.088753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:15.089059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:15.090460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:15.099930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a39886ac-790f-4c74-84c4-53b16662ce02 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:15.101162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 366/728] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:15.101280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a39886ac-790f-4c74-84c4-53b16662ce02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:15.106292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:15.182513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 363/729] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:15.212033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:15.226523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39886ac-790f-4c74-84c4-53b16662ce02 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:15.313727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d454ebe-8333-4219-8895-5cf658db9db4 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:15.342875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:15.351276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:15.352221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:15.352668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:15.353105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:15.364191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:15.367565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:15.389417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:15.390663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:15.391562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:15.391562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:15.391562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:15.393685 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 367/730] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:15.399862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:15.444851 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39886ac-790f-4c74-84c4-53b16662ce02 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:15.507923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:15.517815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a30f6e-5a96-4a8e-b0dc-374d1a41ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:15.519280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a30f6e-5a96-4a8e-b0dc-374d1a41ba34 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:15.525001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a30f6e-5a96-4a8e-b0dc-374d1a41ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:15.525001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a30f6e-5a96-4a8e-b0dc-374d1a41ba34 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:15.525001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a30f6e-5a96-4a8e-b0dc-374d1a41ba34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:15.525001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a30f6e-5a96-4a8e-b0dc-374d1a41ba34 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:15.525445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6fb1-61d8-4dae-9370-de9b9bc2c7e0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:15.565232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:15.565842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:15.566152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:15.566503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:15.567083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:15.567468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:15.569211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:15.584790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 368/731] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:15.585517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:15.660992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 364/732] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:30:15 2022] POST /identity/v3/services => generated 251 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:30:15.662883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a30f6e-5a96-4a8e-b0dc-374d1a41ba34 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:15.697756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3fc17-0a30-4833-b852-dbfacca43fdd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:15.755053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47a21b-7d7f-4abb-b5c8-084c89ae579e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:15.756374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47a21b-7d7f-4abb-b5c8-084c89ae579e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:15.756918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47a21b-7d7f-4abb-b5c8-084c89ae579e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:15.757830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47a21b-7d7f-4abb-b5c8-084c89ae579e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:15.771796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:15.775614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47a21b-7d7f-4abb-b5c8-084c89ae579e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-de47a21b-7d7f-4abb-b5c8-084c89ae579e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:15.780755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:15.785077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 369/733] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:15.786248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47a21b-7d7f-4abb-b5c8-084c89ae579e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:15.811683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a30f6e-5a96-4a8e-b0dc-374d1a41ba34 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:15.821086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:15.858492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235a72-8a65-40c1-914e-27a7cf4228c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:15.859142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235a72-8a65-40c1-914e-27a7cf4228c5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:15.859424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235a72-8a65-40c1-914e-27a7cf4228c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:15.859700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235a72-8a65-40c1-914e-27a7cf4228c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:15.874381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8235a72-8a65-40c1-914e-27a7cf4228c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-a8235a72-8a65-40c1-914e-27a7cf4228c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:15.879261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:15.882112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 365/734] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:15.882949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235a72-8a65-40c1-914e-27a7cf4228c5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:15.930625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47a21b-7d7f-4abb-b5c8-084c89ae579e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:15.970631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:15.971268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:15.971550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:15.971824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:15.980426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:15.981903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:15.996166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:16.002006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 370/735] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:16.003238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.011089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235a72-8a65-40c1-914e-27a7cf4228c5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:16.057206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:16.058124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.058518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.058883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.059671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:16.059980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:16.061398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:16.075777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 366/736] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:16.075777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.083427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:16.104146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:16.134248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294ea02-b90f-442d-9af5-c5e0899131a3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:16.172987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d08a6-afba-4381-9541-c8ae84aa2983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:16.173693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d08a6-afba-4381-9541-c8ae84aa2983 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.174004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d08a6-afba-4381-9541-c8ae84aa2983 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.174302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d08a6-afba-4381-9541-c8ae84aa2983 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.175106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d08a6-afba-4381-9541-c8ae84aa2983 None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:16.175448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d08a6-afba-4381-9541-c8ae84aa2983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:16.176798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d08a6-afba-4381-9541-c8ae84aa2983 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:16.190920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 371/737] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:16.190920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca022db-1a0c-4129-aafc-9280a48c64fa None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.190920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca022db-1a0c-4129-aafc-9280a48c64fa None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.190920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca022db-1a0c-4129-aafc-9280a48c64fa None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.190920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 372/738] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:16.191914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca022db-1a0c-4129-aafc-9280a48c64fa None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.199061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6988d0-cb23-4f7b-b47b-cc17e5ad374c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:16.236441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:16.237079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.237248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.237526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.238297 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:16.238611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:16.240617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:16.259345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 367/739] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 17:30:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:16.260194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.320458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca022db-1a0c-4129-aafc-9280a48c64fa None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:16.344677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:16.361366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:16.361821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0135a8-02b8-4068-845c-ba47cef60abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:16.362498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0135a8-02b8-4068-845c-ba47cef60abf None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.362783 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0135a8-02b8-4068-845c-ba47cef60abf None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.363069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0135a8-02b8-4068-845c-ba47cef60abf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.374858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0135a8-02b8-4068-845c-ba47cef60abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:16.376310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0135a8-02b8-4068-845c-ba47cef60abf None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:16.377261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a674c-dda2-48e6-b828-310900163254 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-5d0135a8-02b8-4068-845c-ba47cef60abf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:16.391997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:16.393173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:16.394106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:16.394106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:16.394106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:16.394106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 373/740] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:16.394702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb32c-ff59-4099-a4de-c7017a620a15 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.394702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb32c-ff59-4099-a4de-c7017a620a15 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.394931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb32c-ff59-4099-a4de-c7017a620a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.414473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c41cb32c-ff59-4099-a4de-c7017a620a15 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:16.415060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c41cb32c-ff59-4099-a4de-c7017a620a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:16.424391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:16.425392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.425728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.426060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.438412 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:16.439866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:16.454909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:16.455872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:16.456755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:16.456755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:16.456755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:16.456755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 368/741] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:16.456998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.502386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 374/742] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:30:16.503161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cb32c-ff59-4099-a4de-c7017a620a15 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.595938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5db12b-b704-4a46-9ea2-9b1eafb81555 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:16.631213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cb32c-ff59-4099-a4de-c7017a620a15 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:16.634034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:16.646605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93a6dd-59fb-474a-8c33-c774469ed43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:16.647217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93a6dd-59fb-474a-8c33-c774469ed43e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.647300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93a6dd-59fb-474a-8c33-c774469ed43e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.650680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93a6dd-59fb-474a-8c33-c774469ed43e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.659945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:16.680264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:16.680264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.680264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.680722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.682320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:16.682528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:16.683868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:16.701645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 375/743] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:16.702912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.757107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93a6dd-59fb-474a-8c33-c774469ed43e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:16.758944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93a6dd-59fb-474a-8c33-c774469ed43e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:16.837121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cff28-1e15-4230-b450-23194d5a9985 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:16.876007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:16.876608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.876889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.877161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.877806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:16.878141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:16.879495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:16.880702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 369/744] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:16 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:30:16.881915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93a6dd-59fb-474a-8c33-c774469ed43e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.897670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 376/745] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:16.897670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:16.931412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:16.933161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:16.997046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93a6dd-59fb-474a-8c33-c774469ed43e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.016157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d078d34-5fb9-40ef-9b6e-1dff2bc035b1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.038211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.038943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.039321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.039662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.047503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:17.049039 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:17.056283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66fe96-2422-4e2f-ad25-2a20570939af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.056837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66fe96-2422-4e2f-ad25-2a20570939af None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.057123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66fe96-2422-4e2f-ad25-2a20570939af None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.057401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66fe96-2422-4e2f-ad25-2a20570939af None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.058081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66fe96-2422-4e2f-ad25-2a20570939af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:17.059432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66fe96-2422-4e2f-ad25-2a20570939af None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:17.061646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:17.063732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 370/746] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:30:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:17.065154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:17.145901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 377/747] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:30:16 2022] POST /identity/v3/services => generated 240 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:17.145901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66fe96-2422-4e2f-ad25-2a20570939af None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:17.183025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:17.188981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b283cf-0afb-486e-9478-2a68abbabb2a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.260262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.261614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.262142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.262613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.277996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:17.279832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:17.298422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:17.299468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:17.300599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:17.300599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:17.300599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:17.305064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:17.308342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 371/748] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:17.309446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:17.342893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66fe96-2422-4e2f-ad25-2a20570939af None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.402515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.403554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.404154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.404619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.421997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:17.425324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:17.464991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:17.467143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 378/749] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:30:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:17.469034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:17.473657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecc511-1daf-4130-8dcd-749ed2e2a209 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.538643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.539594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.540073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.543523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.543523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:17.543523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:17.544612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:17.593681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 372/750] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:30:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:17.593681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:17.601623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:17.691177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4d739-f9d0-4017-952f-4fe5df267274 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.766552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.767372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.767687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.768034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.768815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:17.770318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:17.771753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:17.811387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 379/751] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:17.813501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:17.821423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:17.834342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afa2b8-f5ca-4ad8-9dca-fcbe0ea27ff4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.881130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.882141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.882427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.882720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.883503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:17.883808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:17.885324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:17.917087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 373/752] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 17:30:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:17.918392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:17.954041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3aa0d-7431-4076-a1db-143c6629f6d2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.991322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcfb44-3eca-460a-aff9-486d19cea372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.992048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcfb44-3eca-460a-aff9-486d19cea372 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.992406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcfb44-3eca-460a-aff9-486d19cea372 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.992752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcfb44-3eca-460a-aff9-486d19cea372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.994042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:18.004084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcfb44-3eca-460a-aff9-486d19cea372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:18.006526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcfb44-3eca-460a-aff9-486d19cea372 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-83dcfb44-3eca-460a-aff9-486d19cea372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:18.028108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:18.029117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:18.030040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:18.030040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:18.030040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:18.034475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 380/753] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:18.035898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e8f13-10a7-4568-8f7c-17c612a312fd None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:18.041415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e8f13-10a7-4568-8f7c-17c612a312fd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:18.041415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e8f13-10a7-4568-8f7c-17c612a312fd None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:18.041415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 381/754] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:18.041823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dd8e0-691d-4a1f-adf2-aff4099849df None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:18.041991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e8f13-10a7-4568-8f7c-17c612a312fd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:18.099314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:18.100294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:18.100777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:18.101290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:18.109297 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:18.111097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:18.120133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:18.124818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:18.125689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:18.126741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:18.126741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:18.126741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:18.126741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:18.126741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:18.126741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:18.126741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:18.126741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:18.129161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 374/755] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:18.130200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:18.184162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e8f13-10a7-4568-8f7c-17c612a312fd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:18.223214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bf8fe-b735-4281-8dd2-9c507791f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:18.223891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bf8fe-b735-4281-8dd2-9c507791f601 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:18.224746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bf8fe-b735-4281-8dd2-9c507791f601 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:18.225023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bf8fe-b735-4281-8dd2-9c507791f601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:18.247168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39f8a9-08c1-472d-95f2-1d4c9111b04f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:18.274587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:18.288331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4854d-f658-4351-82bd-723356b2aa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:18.289361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4854d-f658-4351-82bd-723356b2aa9b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:18.289838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4854d-f658-4351-82bd-723356b2aa9b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:18.290323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4854d-f658-4351-82bd-723356b2aa9b None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:18.291180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4854d-f658-4351-82bd-723356b2aa9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:18.291774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4854d-f658-4351-82bd-723356b2aa9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:18.293467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4854d-f658-4351-82bd-723356b2aa9b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:18.306697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 375/756] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:18.308258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9124-a1ae-4f28-afd2-319e0669b5a1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:18.308794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9124-a1ae-4f28-afd2-319e0669b5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:18.309266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9124-a1ae-4f28-afd2-319e0669b5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:18.317799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bf8fe-b735-4281-8dd2-9c507791f601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:18.319002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bf8fe-b735-4281-8dd2-9c507791f601 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:18.329091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-4fda9124-a1ae-4f28-afd2-319e0669b5a1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:18.330699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4fda9124-a1ae-4f28-afd2-319e0669b5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:18.390510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:18.409978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 376/757] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:18.411142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9124-a1ae-4f28-afd2-319e0669b5a1 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:18.448113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 382/758] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:30:18.449119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bf8fe-b735-4281-8dd2-9c507791f601 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:18.532726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9124-a1ae-4f28-afd2-319e0669b5a1 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:18.548777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:18.572965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:18.575639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:18.576143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:18.576689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:18.577601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:18.577794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bf8fe-b735-4281-8dd2-9c507791f601 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:18.578367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:18.580198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:18.594597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 377/759] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:18.595824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:18.625023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:18.625885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:18.626128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:18.626404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:18.636145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:18.638233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:18.652745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:18.653841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:18.654772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:18.654772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:18.654772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:18.654772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:18.654772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:18.654772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:18.654772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:18.654772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:18.660633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 383/760] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:18.661668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:18.691160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:18.715517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d422a-0b6a-43f5-89cd-62407e30513f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:18.759180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:18.760627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:18.761887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:18.762815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:18.787739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:18.790496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:18.849715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:18.851146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:18.852054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:18.852054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:18.852054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:18.855621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 378/761] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:18.855621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:18.888244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:18.967665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e768f-f527-4144-80b3-2538678ef475 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.010546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.011122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.011402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.011665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.012400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=e698e658de56495e8d501ca6dfd75623) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:19.012704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.014188 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.030362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 384/762] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:18 2022] GET /identity/v3/role_assignments?user.id=e698e658de56495e8d501ca6dfd75623&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 259 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:19.031362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:19.046373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4610-e63c-4a45-aeda-68382e7e80ee None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.176311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.176497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.176758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.177127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.180216 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:19.180216 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.186249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.228017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 379/763] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:19.228017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:19.324773 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:19.332692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:19.352795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d7c3e-8253-4295-b168-0b147f07353f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.368431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81493e-6511-4c14-b44a-696f51cd7bf6 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.406525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.409216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.410245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.410966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46b9f4-9d40-4c5b-ab49-206cdd57bedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.411156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.412300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46b9f4-9d40-4c5b-ab49-206cdd57bedd None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.412729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46b9f4-9d40-4c5b-ab49-206cdd57bedd None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.413087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.413203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46b9f4-9d40-4c5b-ab49-206cdd57bedd None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.413960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46b9f4-9d40-4c5b-ab49-206cdd57bedd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:19.414370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46b9f4-9d40-4c5b-ab49-206cdd57bedd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.415854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46b9f4-9d40-4c5b-ab49-206cdd57bedd None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.417247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.430624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 380/764] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:19.436077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3744c-768d-4bd7-9487-1436c445994a None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.436475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3744c-768d-4bd7-9487-1436c445994a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.436859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3744c-768d-4bd7-9487-1436c445994a None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.439617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 381/765] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:19.439617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3744c-768d-4bd7-9487-1436c445994a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:19.445082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:19.446178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:19.447025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 385/766] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 24 17:30:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:19.447899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:19.576705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fed90-f538-43e7-bae4-25ff0475910e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.577787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3744c-768d-4bd7-9487-1436c445994a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.614334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc522-20aa-4f73-8ad3-84a53a4d948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.615273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc522-20aa-4f73-8ad3-84a53a4d948d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.615556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc522-20aa-4f73-8ad3-84a53a4d948d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.615824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc522-20aa-4f73-8ad3-84a53a4d948d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.621909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507ecc8-37e8-4aca-84de-2bf59d0851da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.622571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcc522-20aa-4f73-8ad3-84a53a4d948d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.623080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507ecc8-37e8-4aca-84de-2bf59d0851da None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.623566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507ecc8-37e8-4aca-84de-2bf59d0851da None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.624004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcc522-20aa-4f73-8ad3-84a53a4d948d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.624284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507ecc8-37e8-4aca-84de-2bf59d0851da None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/20dd03dd2fa04147bba827f983a58034/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.626711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507ecc8-37e8-4aca-84de-2bf59d0851da None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=20dd03dd2fa04147bba827f983a58034, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.628239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507ecc8-37e8-4aca-84de-2bf59d0851da None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-84dcc522-20aa-4f73-8ad3-84a53a4d948d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:19.635688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:19.636829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:19.637740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 386/767] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:19.638253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4528819-94ef-47f7-b6b1-5820e6407c06 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.638814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4528819-94ef-47f7-b6b1-5820e6407c06 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.638814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4528819-94ef-47f7-b6b1-5820e6407c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.646387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:19.657516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d4528819-94ef-47f7-b6b1-5820e6407c06 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:19.658823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d4528819-94ef-47f7-b6b1-5820e6407c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:19.726894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 387/768] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:19.727739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4528819-94ef-47f7-b6b1-5820e6407c06 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:19.810234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 382/769] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:19 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/20dd03dd2fa04147bba827f983a58034/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:19.811334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507ecc8-37e8-4aca-84de-2bf59d0851da None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:19.823512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:19.854917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4528819-94ef-47f7-b6b1-5820e6407c06 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.903110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.904444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.904953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.905427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.906702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:19.907472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.909341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.912855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:19.917417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 388/770] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 17:30:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:19.923603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.056146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507ecc8-37e8-4aca-84de-2bf59d0851da None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.089801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed10b0a-1b0b-4845-b106-4adc36af9dc2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.117877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.117877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.117877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.117877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.118266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:20.118266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.121875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.141883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.141883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.141883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.141883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.142305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:20.142305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.142430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 383/771] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:20.142502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.142581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.151524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 389/772] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 17:30:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:20.152427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.229880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:20.233874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:20.270292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e00a79-1ca6-4f35-a5e1-f1e0fd0e2b17 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.281014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d823c-9a1f-48e2-83b8-59c6126413f2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.307618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.308283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.308628 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.308965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] PATH_INFO: `/v3/services/network` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.309617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.311119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.325349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:20.325543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:20.326514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:20.327427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:20.328169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.328169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.328169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.328169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=364e084e459d4263815ab59720d039e4) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:20.328169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.328559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 384/773] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:20.329239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.333872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.341261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 390/774] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/role_assignments?user.id=364e084e459d4263815ab59720d039e4&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:20.342161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.476300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1dbd5-fef8-4e65-bc93-e166997dc37f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.510897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:20.522571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.524733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.524733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.524733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.525327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] RBAC: Adding query filter params (name=network) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:20.525815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.528254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.534329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 391/775] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:20.543101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.602253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a51b95-74ad-41f2-a27b-1f313675b49f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.632201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:20.642754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.651100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.651100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.651100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.651100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] RBAC: Adding query filter params (interface=public, service_id=ca5cd6b6c11448c4bf92ea6304f31c71, region_id=RegionOne) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:20.651100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.651100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.656653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 385/776] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/endpoints?service_id=ca5cd6b6c11448c4bf92ea6304f31c71&interface=public®ion_id=RegionOne => generated 194 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:20.666928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.679056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7fc23-cf04-4ad4-b441-a112a19a5519 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.722978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.725787 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.726206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.727213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.730827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] RBAC: Adding query filter params (type=network) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:20.731474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.733881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.751748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 392/777] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:20.755491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.870855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:20.909888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9d370-9453-4950-a2e0-ad78d444feab None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.983104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49600b0-05b8-4d1b-992a-ad7b02c08777 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.993817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4731626-7e87-45b9-aabd-29bdfdd1f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.994455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4731626-7e87-45b9-aabd-29bdfdd1f706 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.994735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4731626-7e87-45b9-aabd-29bdfdd1f706 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.995002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4731626-7e87-45b9-aabd-29bdfdd1f706 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.995644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4731626-7e87-45b9-aabd-29bdfdd1f706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.997145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4731626-7e87-45b9-aabd-29bdfdd1f706 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:21.005878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 386/778] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/services => generated 772 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:21.044348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef14b3-14ac-4550-b998-49391fc84101 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:21.044714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef14b3-14ac-4550-b998-49391fc84101 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:21.044916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef14b3-14ac-4550-b998-49391fc84101 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:21.046914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 387/779] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:21.053397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a45fd-e250-48c8-93c0-7b4891b18698 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:21.053732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a45fd-e250-48c8-93c0-7b4891b18698 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:21.053992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a45fd-e250-48c8-93c0-7b4891b18698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:21.057747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:21.059047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:21.059334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:21.059614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:21.060376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] RBAC: Adding query filter params (interface=public, service_id=d7e9413d81c3418aba7d9cc41a62af63, region_id=RegionOne) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:21.060692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:21.062223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:21.086733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 393/780] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/endpoints?service_id=d7e9413d81c3418aba7d9cc41a62af63&interface=public®ion_id=RegionOne => generated 194 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:21.087732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e01a45fd-e250-48c8-93c0-7b4891b18698 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:21.089965 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:21.090641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e01a45fd-e250-48c8-93c0-7b4891b18698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:21.203902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:21.232987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 388/781] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:21.274058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf7b7f-d013-41c0-a362-dbcf80faf5bc None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:21.293419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:21.310788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401e45d-5607-48ab-b09b-b5508c4a59a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:21.311407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401e45d-5607-48ab-b09b-b5508c4a59a9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:21.311684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401e45d-5607-48ab-b09b-b5508c4a59a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:21.312049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401e45d-5607-48ab-b09b-b5508c4a59a9 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:21.313276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401e45d-5607-48ab-b09b-b5508c4a59a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:21.314318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401e45d-5607-48ab-b09b-b5508c4a59a9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:21.320616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 394/782] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:30:21 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:21.539899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a45fd-e250-48c8-93c0-7b4891b18698 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:21.662134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a45fd-e250-48c8-93c0-7b4891b18698 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:21.714611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f5777-ca7a-4706-ad59-5134493f999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:21.715309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f5777-ca7a-4706-ad59-5134493f999a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:21.715593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f5777-ca7a-4706-ad59-5134493f999a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:21.715865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f5777-ca7a-4706-ad59-5134493f999a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:21.731522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f5777-ca7a-4706-ad59-5134493f999a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-2f3f5777-ca7a-4706-ad59-5134493f999a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:21.736481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:21.737417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:21.738935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 389/783] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:21.746975 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401e45d-5607-48ab-b09b-b5508c4a59a9 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:21.909271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401e45d-5607-48ab-b09b-b5508c4a59a9 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:21.963951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f617e-a4c4-4f43-ba04-839be43d437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:21.964572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f617e-a4c4-4f43-ba04-839be43d437c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:21.964860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f617e-a4c4-4f43-ba04-839be43d437c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:21.965146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f617e-a4c4-4f43-ba04-839be43d437c None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:21.965818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f617e-a4c4-4f43-ba04-839be43d437c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:21.976535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f617e-a4c4-4f43-ba04-839be43d437c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:21.978012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f617e-a4c4-4f43-ba04-839be43d437c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:21.987214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 395/784] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:21.993918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f5777-ca7a-4706-ad59-5134493f999a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:22.003862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d6582-f615-4dfb-a3ed-f42528cbc064 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:22.004157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d6582-f615-4dfb-a3ed-f42528cbc064 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:22.004456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d6582-f615-4dfb-a3ed-f42528cbc064 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:22.005946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 396/785] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:22.011336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7860-361e-4675-b89c-5d159ebbdb31 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:22.011621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7860-361e-4675-b89c-5d159ebbdb31 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:22.011900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7860-361e-4675-b89c-5d159ebbdb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:22.043588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3bae7860-361e-4675-b89c-5d159ebbdb31 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:22.044780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3bae7860-361e-4675-b89c-5d159ebbdb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:22.055344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:22.106469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:22.118047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f5777-ca7a-4706-ad59-5134493f999a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:22.163027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:22.164013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:22.164634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 397/786] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:22.165014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:22.165494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:22.181336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:22.183157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:22.199568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:22.200551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:22.201476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:22.201476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:22.201476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:22.203531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 390/787] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:22.210952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae7860-361e-4675-b89c-5d159ebbdb31 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:22.371805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae7860-361e-4675-b89c-5d159ebbdb31 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:22.429917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:22.430485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:22.430765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:22.431041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:22.431706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:22.432011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:22.433404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:22.446250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 398/788] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:22.451826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:22.458370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:22.521926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:22.664907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:22.671433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc830fd-e3da-4f13-988b-75eb32e511a6 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:22.675140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4b1c7-49e8-48a4-913b-8ebdf1ab7c11 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:22.748686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e26f6-fc0f-4f60-9e45-088bfaf03760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:22.750637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e26f6-fc0f-4f60-9e45-088bfaf03760 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:22.751505 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52473ab-447f-430a-a6a1-4137e8da4382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:22.752350 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e26f6-fc0f-4f60-9e45-088bfaf03760 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:22.752861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e26f6-fc0f-4f60-9e45-088bfaf03760 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:22.753694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52473ab-447f-430a-a6a1-4137e8da4382 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:22.755210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52473ab-447f-430a-a6a1-4137e8da4382 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:22.757503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52473ab-447f-430a-a6a1-4137e8da4382 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:22.771673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e26f6-fc0f-4f60-9e45-088bfaf03760 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:22.778674 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e26f6-fc0f-4f60-9e45-088bfaf03760 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:22.781678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52473ab-447f-430a-a6a1-4137e8da4382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e52473ab-447f-430a-a6a1-4137e8da4382 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:22.788534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:22.789571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:22.790454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:22.794323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 399/789] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:22.795775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355320d5-2e94-4179-b504-c54a09f27f61 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:22.796060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355320d5-2e94-4179-b504-c54a09f27f61 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:22.796327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355320d5-2e94-4179-b504-c54a09f27f61 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-855e26f6-fc0f-4f60-9e45-088bfaf03760 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:22.797518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:22.798467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:22.799432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:22.800146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 400/790] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:22.802154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 391/791] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:22.804544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355320d5-2e94-4179-b504-c54a09f27f61 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:22.808286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f371c-422a-44c5-ade1-c4c8d2916bc3 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:22.809471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f371c-422a-44c5-ade1-c4c8d2916bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:22.809471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f371c-422a-44c5-ade1-c4c8d2916bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:22.830496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a29f371c-422a-44c5-ade1-c4c8d2916bc3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:22.831852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a29f371c-422a-44c5-ade1-c4c8d2916bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:22.907252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:22.917877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 392/792] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:22.917877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f371c-422a-44c5-ade1-c4c8d2916bc3 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:22.930955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355320d5-2e94-4179-b504-c54a09f27f61 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.003722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.004264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.004545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.004817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.006289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:23.006591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:23.014250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:23.038438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 401/793] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:23.044252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.069780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:23.088662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f371c-422a-44c5-ade1-c4c8d2916bc3 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.173878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.174175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.174928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.176714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.180338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:23.182694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:23.196331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:23.244169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 393/794] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 17:30:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:23.265163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.310512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe428-1ab2-4a58-b967-75fb5e4e2b9f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.359572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.360490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.361104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.361525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.378518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:23.381157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:23.403097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:23.404256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:23.405158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:23.405158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:23.405158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:23.412557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 402/795] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:23.415406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.430535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:23.436353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f174-169f-404a-b0a9-a3db640dda21 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.479413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.480365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.480815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.481378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.495435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:23.497258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:23.508373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:23.528052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:23.529061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:23.529969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:23.529969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:23.529969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:23.530668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 394/796] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:23.531796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.550776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4007da1-59f1-428a-b9f0-573a4e8f3091 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.604752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a8a6f-0190-4d47-bcf2-d11999ec0fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.606012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8a6f-0190-4d47-bcf2-d11999ec0fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.606297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8a6f-0190-4d47-bcf2-d11999ec0fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.606573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8a6f-0190-4d47-bcf2-d11999ec0fb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.634771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a8a6f-0190-4d47-bcf2-d11999ec0fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-122a8a6f-0190-4d47-bcf2-d11999ec0fb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:23.640748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:23.641714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:23.642575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:23.652040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 403/797] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:23.654026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a8a6f-0190-4d47-bcf2-d11999ec0fb4 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.675909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eeaa6c-00fd-4fa2-8320-18d7a46067fd None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.722909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.724329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.724775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.725169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.725977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:23.726357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:23.727935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:23.732998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:23.742045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 395/798] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:23.742953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.776949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a8a6f-0190-4d47-bcf2-d11999ec0fb4 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.807163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:23.814302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1ff57-2258-40bf-b126-dba646945b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.820005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1ff57-2258-40bf-b126-dba646945b7a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.820005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1ff57-2258-40bf-b126-dba646945b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.820005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1ff57-2258-40bf-b126-dba646945b7a None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.820005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1ff57-2258-40bf-b126-dba646945b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:23.820005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1ff57-2258-40bf-b126-dba646945b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:23.820507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1ff57-2258-40bf-b126-dba646945b7a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:23.836500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 404/799] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:23.837938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c26f6-666b-461b-a15f-1790845d2fd4 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.838224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c26f6-666b-461b-a15f-1790845d2fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.838803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c26f6-666b-461b-a15f-1790845d2fd4 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.844228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 405/800] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:23.845452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524624f-9c72-4abe-a7d4-fe024aaf6404 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.845741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524624f-9c72-4abe-a7d4-fe024aaf6404 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.846038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524624f-9c72-4abe-a7d4-fe024aaf6404 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.847368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 406/801] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:23.848097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524624f-9c72-4abe-a7d4-fe024aaf6404 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.857875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7c55-859c-4a40-9410-30adc6de78db None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.901030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.901793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.902097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.902371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.903096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:23.903331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:23.904817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:23.914644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 396/802] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:23.915312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.964758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524624f-9c72-4abe-a7d4-fe024aaf6404 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.992453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.000488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c72441-3795-4e7f-88a7-60a4188d8d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.001566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c72441-3795-4e7f-88a7-60a4188d8d8d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.001566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c72441-3795-4e7f-88a7-60a4188d8d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.001805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c72441-3795-4e7f-88a7-60a4188d8d8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.009630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c72441-3795-4e7f-88a7-60a4188d8d8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.011125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c72441-3795-4e7f-88a7-60a4188d8d8d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-d3c72441-3795-4e7f-88a7-60a4188d8d8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:24.024914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:24.026034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:24.027009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 407/803] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:24.027599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f20a8-8c56-4cba-a240-4136a2fd7776 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.027599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f20a8-8c56-4cba-a240-4136a2fd7776 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.027769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f20a8-8c56-4cba-a240-4136a2fd7776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.034514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc79ba3-b70c-4a0a-bc1e-54bf26744cd2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.045782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f09f20a8-8c56-4cba-a240-4136a2fd7776 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:24.047035 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f09f20a8-8c56-4cba-a240-4136a2fd7776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:24.059500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.074721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7992a46-1cb6-41c9-8dec-deb87201ef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.075396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7992a46-1cb6-41c9-8dec-deb87201ef6d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.075725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7992a46-1cb6-41c9-8dec-deb87201ef6d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.076000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7992a46-1cb6-41c9-8dec-deb87201ef6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.083908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7992a46-1cb6-41c9-8dec-deb87201ef6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.085781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7992a46-1cb6-41c9-8dec-deb87201ef6d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-f7992a46-1cb6-41c9-8dec-deb87201ef6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:24.097054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:24.098248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:24.099218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:24.099218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:24.099218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:24.099218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:24.099218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:24.099218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:24.099218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:24.099218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:24.099854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 397/804] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:24.100770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c5813-1ded-415b-89a3-1f749bdcf7dc None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.101052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c5813-1ded-415b-89a3-1f749bdcf7dc None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.101325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c5813-1ded-415b-89a3-1f749bdcf7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.119639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-1a1c5813-1ded-415b-89a3-1f749bdcf7dc None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:24.120988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1a1c5813-1ded-415b-89a3-1f749bdcf7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:24.123011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 408/805] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:24.123984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f20a8-8c56-4cba-a240-4136a2fd7776 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:24.215214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 398/806] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:24.215214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c5813-1ded-415b-89a3-1f749bdcf7dc None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:24.241164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f20a8-8c56-4cba-a240-4136a2fd7776 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.261408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.287887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.288587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.288871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.289144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.301428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.304128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:24.322356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:24.323326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:24.324263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:24.324263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:24.324263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:24.328592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 409/807] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:24.329477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:24.350690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c5813-1ded-415b-89a3-1f749bdcf7dc None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.366035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.400525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.401144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.401812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.402302 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.403331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:24.404077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.405612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.423074 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 399/808] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:30:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:24.428834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:24.448709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9437f4f-861b-4cb1-8878-0d1560eea404 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.488272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.488884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.489167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.490171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.490781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:24.491088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.492508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.505123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 410/809] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:24.505996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:24.535881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.538494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79cf9-b33c-487b-bb44-f8e4520c6bac None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.616286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0a346-434d-43b3-8553-7617a4f22d29 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.652590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.653230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.653509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.653783 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.654409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.655779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:24.665833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:24.666994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:30:24.667844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:24.668692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 411/810] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 24 17:30:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:24.668763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:24.696755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.697357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.697629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.697921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.698587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:24.698883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.700236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.717514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 400/811] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:24.717514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:24.742912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.769777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.776532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf52033-08a0-4775-a1b4-a3e6183c5bb0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.810903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fbe5a-8b88-4397-8cea-75e19005c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.811598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fbe5a-8b88-4397-8cea-75e19005c863 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.811790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fbe5a-8b88-4397-8cea-75e19005c863 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.812061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fbe5a-8b88-4397-8cea-75e19005c863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.816333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ca7de-9388-4e36-aee8-257d2746c105 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.853214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:24.853835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:24.854140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:24.854434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] PATH_INFO: `/v3/services/network` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:24.855018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.856370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:24.864998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:24.866547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:24.867633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 401/812] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:24 2022] GET /identity/v3/services/network => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:24.868442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:24.885069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fbe5a-8b88-4397-8cea-75e19005c863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:24.886544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fbe5a-8b88-4397-8cea-75e19005c863 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:24.964069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.972712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25032ca-86af-400a-a336-567a72656c4d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:24.987175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:24.994452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 412/813] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:30:24.995290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fbe5a-8b88-4397-8cea-75e19005c863 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.011840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618916e-a266-4551-9df7-cd9b08376783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.012539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618916e-a266-4551-9df7-cd9b08376783 None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.012827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618916e-a266-4551-9df7-cd9b08376783 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.013095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618916e-a266-4551-9df7-cd9b08376783 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/e698e658de56495e8d501ca6dfd75623/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.015150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618916e-a266-4551-9df7-cd9b08376783 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=e698e658de56495e8d501ca6dfd75623, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.016586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618916e-a266-4551-9df7-cd9b08376783 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.094670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fbe5a-8b88-4397-8cea-75e19005c863 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.112310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 402/814] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:24 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/e698e658de56495e8d501ca6dfd75623/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:25.113155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618916e-a266-4551-9df7-cd9b08376783 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.140622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.140977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.141925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.141925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.142209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:25.142501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.143910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.149957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 413/815] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 17:30:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:25.155440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.228139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:25.236665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618916e-a266-4551-9df7-cd9b08376783 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.287234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.287876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.288155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.288424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.289885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc6b5-4b25-4709-93a6-df70f9028fac None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.296770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.298316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:25.310434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:25.311430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:25.312384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:25.314181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 403/816] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:30:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:25.314981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.334334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.334575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:25.334951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.335235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.335504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.336144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] RBAC: Adding query filter params (name=network) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:25.336472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.337816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.342598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 414/817] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:25.343380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.434205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f2ee4-4e4c-461a-8a15-54dd99540a47 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.456071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60c8c-309d-4f70-82f5-224feda54407 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.474588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.475347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.475703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.475984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.483158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:25.486968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.488416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.496800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.498235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.498235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.498235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.499023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:25.499325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.500673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:25.505336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:25.506320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:25.507375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:25.507375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:25.507375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:25.510371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 415/818] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 17:30:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:25.511099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 404/819] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:25.511429 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.511859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.584854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:25.626138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad221c-63e0-480e-96b3-d5fb7065856b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.627245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84917ef-107e-4ea6-afad-c320068a1948 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.664680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.665347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.665634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.665936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.666578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:25.666888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.668351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.672971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.673936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 405/820] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:25.679282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.679375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.679375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.679375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.679584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:25.679784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.681314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.702884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 416/821] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 17:30:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:25.704804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.764828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:25.791289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b5161-f836-448b-88fa-01172fbb50e1 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.822256 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:25.828755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4632a6-61da-43aa-92c3-583244d85cef None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:25.831310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda35fc6-0524-41fc-acaa-0bfc6aec85af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.831915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda35fc6-0524-41fc-acaa-0bfc6aec85af None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.832192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda35fc6-0524-41fc-acaa-0bfc6aec85af None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.832462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda35fc6-0524-41fc-acaa-0bfc6aec85af None admin] PATH_INFO: `/v3/endpoints` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.833290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda35fc6-0524-41fc-acaa-0bfc6aec85af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.834685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda35fc6-0524-41fc-acaa-0bfc6aec85af None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.865912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:25.866581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:25.866906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:25.867143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:25.867799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:25.868103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:25.869471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:25.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 417/822] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:25.882488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.986102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 406/823] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:25.987520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda35fc6-0524-41fc-acaa-0bfc6aec85af None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:25.999583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d1039-047e-4a8d-8085-93d450f6c26b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:26.002692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:26.064158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb4a5-5852-4226-9364-f90ea91d3d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:26.065245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb4a5-5852-4226-9364-f90ea91d3d09 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:26.065737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb4a5-5852-4226-9364-f90ea91d3d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:26.066972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb4a5-5852-4226-9364-f90ea91d3d09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:26.068149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdb4a5-5852-4226-9364-f90ea91d3d09 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:26.070308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdb4a5-5852-4226-9364-f90ea91d3d09 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:26.127240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda35fc6-0524-41fc-acaa-0bfc6aec85af None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:26.167027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:26.167819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:26.168200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:26.168577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:26.181196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:26.182888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 418/824] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:25 2022] POST /identity/v3/endpoints => generated 337 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:26.183009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:26.184012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb4a5-5852-4226-9364-f90ea91d3d09 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:26.200969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:26.203420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:26.204339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:26.204339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:26.204339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:26.204339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 407/825] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:26.210306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:26.271595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:26.354933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb4a5-5852-4226-9364-f90ea91d3d09 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:26.363932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c5153-7143-4569-8da8-8ee628169c12 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:26.405784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:26.406730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:26.407155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:26.407610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:26.408270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:26.408573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:26.410032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:26.423608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 408/826] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:26.431157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:26.446817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:26.447449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:26.449624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:26.449624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:26.456303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:26.458407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:26.475700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:26.476643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:26.477530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:26.477530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:26.477530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:26.477530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:26.477530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:26.477530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:26.477530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:26.477530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:26.483063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 419/827] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:26.486171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:26.510008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:26.570068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:26.576726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb0285-b5d4-4d7d-aaa3-8ea5824bfd10 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:26.621096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:26.622314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:26.622807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:26.623300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:26.628034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ef684-cca7-4bea-beab-42590fbf2de9 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:26.639419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:26.641373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:26.660258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:26.661818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:26.662791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:26.662791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:26.662791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:26.662791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:26.662791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:26.662791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:26.662791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:26.662791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:26.670190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 409/828] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:26.671867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:26.673337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:26.673974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:26.674258 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:26.674534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:26.675290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:26.675691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:26.678641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:26.692448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 420/829] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:26.697982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:26.774850 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:26.795313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b028-5cee-4050-95e0-d0cd24de4c9c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:26.810219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fc86b-b5ba-46b0-bfcf-18b560c19380 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:26.836363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02525dd8-143f-481b-a9af-7427b5140404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:26.837311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02525dd8-143f-481b-a9af-7427b5140404 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:26.837825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02525dd8-143f-481b-a9af-7427b5140404 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:26.838428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02525dd8-143f-481b-a9af-7427b5140404 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:26.839436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02525dd8-143f-481b-a9af-7427b5140404 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:26.839940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02525dd8-143f-481b-a9af-7427b5140404 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:26.841780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02525dd8-143f-481b-a9af-7427b5140404 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:26.850652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:26.851277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:26.851542 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:26.851809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:26.852600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=20dd03dd2fa04147bba827f983a58034) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:26.852918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:26.854381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:26.869893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 421/830] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:26 2022] GET /identity/v3/role_assignments?user.id=20dd03dd2fa04147bba827f983a58034&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:26.870102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 410/831] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:26.873233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:27.000860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5587-7261-4898-bf67-a0fefde873d0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:27.043756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec33b46-a835-4dd9-b08c-563193e45886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:27.044522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec33b46-a835-4dd9-b08c-563193e45886 None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:27.044904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec33b46-a835-4dd9-b08c-563193e45886 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:27.045247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec33b46-a835-4dd9-b08c-563193e45886 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/364e084e459d4263815ab59720d039e4/roles/39294eb951d24b04be9b4b5d98b36640` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:27.048979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:27.062232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec33b46-a835-4dd9-b08c-563193e45886 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=364e084e459d4263815ab59720d039e4, role_id=39294eb951d24b04be9b4b5d98b36640)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:27.065067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec33b46-a835-4dd9-b08c-563193e45886 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:27.154461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6305d-a067-42a7-8cf8-9efb2095367d None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:27.154824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6305d-a067-42a7-8cf8-9efb2095367d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:27.155477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6305d-a067-42a7-8cf8-9efb2095367d None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:27.157262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 411/832] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:27.158250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 422/833] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:26 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/364e084e459d4263815ab59720d039e4/roles/39294eb951d24b04be9b4b5d98b36640 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:27.163615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3bd40-68e2-476b-88a0-f8c8acb46505 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:27.164036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3bd40-68e2-476b-88a0-f8c8acb46505 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:27.164408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3bd40-68e2-476b-88a0-f8c8acb46505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:27.184798 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-29d3bd40-68e2-476b-88a0-f8c8acb46505 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:27.186302 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-29d3bd40-68e2-476b-88a0-f8c8acb46505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:27.200090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:27.275991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 412/834] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:27.559767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec33b46-a835-4dd9-b08c-563193e45886 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:27.669975 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec33b46-a835-4dd9-b08c-563193e45886 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:27.709562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6a8ae-ebc7-4b11-87bc-c8121f0e029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:27.710236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6a8ae-ebc7-4b11-87bc-c8121f0e029a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:27.710518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6a8ae-ebc7-4b11-87bc-c8121f0e029a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:27.710803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6a8ae-ebc7-4b11-87bc-c8121f0e029a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:27.721547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6a8ae-ebc7-4b11-87bc-c8121f0e029a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-dca6a8ae-ebc7-4b11-87bc-c8121f0e029a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:27.726507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:27.727658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:27.728686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 423/835] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:27.733961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3bd40-68e2-476b-88a0-f8c8acb46505 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:27.845042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3bd40-68e2-476b-88a0-f8c8acb46505 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:27.882154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:27.883009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:27.883519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:27.883961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:27.884777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:27.885220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:27.886892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:27.896906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 413/836] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:27.905502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6a8ae-ebc7-4b11-87bc-c8121f0e029a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:27.972131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:28.017418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6a8ae-ebc7-4b11-87bc-c8121f0e029a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:28.056240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6944528-3c91-4461-b1ba-58cfe223e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:28.056866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944528-3c91-4461-b1ba-58cfe223e039 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.057142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944528-3c91-4461-b1ba-58cfe223e039 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.057409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944528-3c91-4461-b1ba-58cfe223e039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.068087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6944528-3c91-4461-b1ba-58cfe223e039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:28.069592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6944528-3c91-4461-b1ba-58cfe223e039 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-d6944528-3c91-4461-b1ba-58cfe223e039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:28.085906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:28.087148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:28.088231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:28.088231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:28.088231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:28.088231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 424/837] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:28.093893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:28.121978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:28.191303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e6119-545e-456c-a04a-fde6b4ab093d None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.191889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e6119-545e-456c-a04a-fde6b4ab093d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.192202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e6119-545e-456c-a04a-fde6b4ab093d None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.194488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 425/838] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:28.199077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb4c8-d5e1-479f-8c4b-0e8b5b80b053 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.199521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb4c8-d5e1-479f-8c4b-0e8b5b80b053 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.199899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb4c8-d5e1-479f-8c4b-0e8b5b80b053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.215192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69c003-7b48-4466-adad-6c04ac5f7894 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:28.224313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-534fb4c8-d5e1-479f-8c4b-0e8b5b80b053 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:28.226052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-534fb4c8-d5e1-479f-8c4b-0e8b5b80b053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:28.262184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:28.263004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.265229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.265229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.265229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:28.265471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:28.267515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:28.286263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 414/839] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:28.290735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:28.310820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 426/840] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:28.373725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:28.409067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c9912-dd06-411e-b930-b78a1bf1e499 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:28.446642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:28.447606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.448080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.448523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.456464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:28.458379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:28.471391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:28.472395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:30:28.474050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:28.476641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 415/841] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:28.482606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb4c8-d5e1-479f-8c4b-0e8b5b80b053 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:28.517086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:28.582148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:28.607538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb4c8-d5e1-479f-8c4b-0e8b5b80b053 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:28.660012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:28.662969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.663442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.663893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.665029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:28.665482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:28.668219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:28.697333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 427/842] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 17:30:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:28.699881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:28.709312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3deea3-f30c-4e6e-8718-cb1693de30d8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:28.739823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:28.748467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:28.754747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.754747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.754747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.754747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:28.754747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:28.762218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:28.763204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:28.764060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:28.768261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 416/843] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:28.769518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:28.821249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:28.838698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b293b-62d6-4e78-bc5b-d138b234fd44 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:28.877453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73db88-4723-457c-9252-a53c77463be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:28.878397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73db88-4723-457c-9252-a53c77463be9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.878853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73db88-4723-457c-9252-a53c77463be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.879324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73db88-4723-457c-9252-a53c77463be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.884931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586271c5-84be-48fa-9109-36df17d2ae68 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:28.923309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:28.924037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:28.924523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:28.924896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:28.925887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:28.926203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:28.927752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:28.938269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 417/844] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:28.939976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:28.974421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73db88-4723-457c-9252-a53c77463be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:28.976639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73db88-4723-457c-9252-a53c77463be9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:29.002010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:29.054816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1e2a-a9ae-46d7-86d0-005e31243a39 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:29.072006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:29.089252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 428/845] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:30:29.091871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cad5a-f661-4787-ae7b-321cffb8f85b None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.092281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cad5a-f661-4787-ae7b-321cffb8f85b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.092652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cad5a-f661-4787-ae7b-321cffb8f85b None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.099063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 429/846] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:29.099063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cad5a-f661-4787-ae7b-321cffb8f85b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:29.099485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0185a7-1e57-4d42-b3e0-44a9d46e4b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:29.100128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0185a7-1e57-4d42-b3e0-44a9d46e4b44 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.100386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0185a7-1e57-4d42-b3e0-44a9d46e4b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.100659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0185a7-1e57-4d42-b3e0-44a9d46e4b44 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.101296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0185a7-1e57-4d42-b3e0-44a9d46e4b44 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:29.101593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0185a7-1e57-4d42-b3e0-44a9d46e4b44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:29.102986 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0185a7-1e57-4d42-b3e0-44a9d46e4b44 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:29.115396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 418/847] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:29.115396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efba18-50dd-41ab-bfe7-8aa18a1ac2d5 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.115396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efba18-50dd-41ab-bfe7-8aa18a1ac2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.115396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efba18-50dd-41ab-bfe7-8aa18a1ac2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.129491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e3efba18-50dd-41ab-bfe7-8aa18a1ac2d5 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:29.131603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e3efba18-50dd-41ab-bfe7-8aa18a1ac2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:29.222861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cad5a-f661-4787-ae7b-321cffb8f85b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:29.229929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 419/848] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:29.229929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb83054-6fb7-4efa-8c0a-c3889648964e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.229929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb83054-6fb7-4efa-8c0a-c3889648964e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.229929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb83054-6fb7-4efa-8c0a-c3889648964e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.229929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 420/849] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:29.232884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6a827-ccea-4e89-970c-17ed8d47dfc1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.233188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6a827-ccea-4e89-970c-17ed8d47dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.233470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6a827-ccea-4e89-970c-17ed8d47dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.264595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0ee6a827-ccea-4e89-970c-17ed8d47dfc1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:29.266116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0ee6a827-ccea-4e89-970c-17ed8d47dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:29.272964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:29.274366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.274793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.275225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.288469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:29.291022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:29.309336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:29.310432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:29.311321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:29.311321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:29.311321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:29.314070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 430/850] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:29.316554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:29.341465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:29.368565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:29.376697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 421/851] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:29.436861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c11ebe-0b79-4130-b689-9323c2799469 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:29.480058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:29.481133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.481588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.482078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.482899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:29.483397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:29.484969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:29.499442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 431/852] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:29.506974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6a827-ccea-4e89-970c-17ed8d47dfc1 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:29.511586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:29.634713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:29.638299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6a827-ccea-4e89-970c-17ed8d47dfc1 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:29.638433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1355b9-bd30-470a-9093-eeff94eb09ac None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:29.674849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:29.675511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.675794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.676068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.676701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:29.678105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:29.683524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:29.684415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.684893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.685381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:29.686867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:29.688110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:29.688982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 432/853] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 24 17:30:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:29.689785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:29.696947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:29.698704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:29.711053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:29.711993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:29.712899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 422/854] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:29.713413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:29.763836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:29.805647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cb6c0-0ada-4b13-a5c2-469119666307 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:29.829786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912427d6-0b28-4bdd-8543-f9e8976ee5b6 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:29.844170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a15ba0-adba-42ce-83a4-c6a43cb0d585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:29.844801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a15ba0-adba-42ce-83a4-c6a43cb0d585 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.845079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a15ba0-adba-42ce-83a4-c6a43cb0d585 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.845350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a15ba0-adba-42ce-83a4-c6a43cb0d585 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.857340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a15ba0-adba-42ce-83a4-c6a43cb0d585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e5a15ba0-adba-42ce-83a4-c6a43cb0d585 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:29.862579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:29.863495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:29.864463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 433/855] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:29.865229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a15ba0-adba-42ce-83a4-c6a43cb0d585 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:29.872786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:29.873474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:29.873830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:29.874204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:29.874915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:29.875345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:29.876771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:29.882172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 423/856] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:29.887420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:29.892221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:29.984349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a15ba0-adba-42ce-83a4-c6a43cb0d585 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:30.005763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17775ba-0d73-4d85-9c4a-42c4b2c5116a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:30.009137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:30.023899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:30.024445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:30.024723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:30.024996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:30.025705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:30.026053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:30.027478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:30.045638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 434/857] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:30.045638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:30.053326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:30.053972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:30.054257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:30.054535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:30.055212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:30.055517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:30.056880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:30.065893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 424/858] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:30.066613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:30.140450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:30.159428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970fac-3ec4-479d-b29c-f8bca4fa40f6 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:30.173807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd75157-d33c-43b0-8c49-97c27cad937d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:30.199126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:30.199734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:30.200020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:30.200296 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:30.200951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:30.201230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:30.202623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:30.207533 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 435/859] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:30.208900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:30.219200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:30.220135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:30.220588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:30.221050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:30.221974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=e698e658de56495e8d501ca6dfd75623) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:30.222359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:30.224608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:30.231995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 425/860] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:30 2022] GET /identity/v3/role_assignments?user.id=e698e658de56495e8d501ca6dfd75623&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:30.356807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180114e4-ea0f-40ac-8e45-965b77aeef75 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:30.416780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:30.417710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:30.418329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:30.418798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:30.431195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:30.433051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:30.440301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:30.449648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:30.456752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:30.458290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:30.458290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:30.458290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:30.458290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 436/861] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:30.458540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:30.497830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:30.573725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4c059-8d44-41e1-98e0-46662e95d1bf None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:30.610829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:30.611762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:30.612183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:30.612596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:30.613382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:30.613981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:30.615518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:30.627950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 426/862] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:30.634243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:30.745183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7060b-9483-4dcc-b3c6-eb28f4cfc0a5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:30.782351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:30.783143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:30.783514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:30.783878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:30.792753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:30.794570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:30.807201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:30.808193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:30.809162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 437/863] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:30:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:30.819061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:30.930254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e344eed-5c6f-4041-be36-b2069f630bc0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:30.935810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:30.972436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453f8ca-83e5-4ef3-8617-7eda62507af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:30.973574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453f8ca-83e5-4ef3-8617-7eda62507af6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:30.974141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453f8ca-83e5-4ef3-8617-7eda62507af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:30.974615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453f8ca-83e5-4ef3-8617-7eda62507af6 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:30.975640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453f8ca-83e5-4ef3-8617-7eda62507af6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:30.976159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453f8ca-83e5-4ef3-8617-7eda62507af6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:30.977770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453f8ca-83e5-4ef3-8617-7eda62507af6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:30.997997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 427/864] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 17:30:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:31.003964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:31.066718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:31.110333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecaf69-a66f-49c5-b293-985913d626e3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:31.139574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f270ba-27ad-4fe9-bd2f-9b5601753151 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:31.139873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f270ba-27ad-4fe9-bd2f-9b5601753151 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:31.140130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f270ba-27ad-4fe9-bd2f-9b5601753151 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:31.141743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 428/865] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:31.147359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93856ca8-0387-458b-8018-0156b9cb3eb9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:31.147646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93856ca8-0387-458b-8018-0156b9cb3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:31.147919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93856ca8-0387-458b-8018-0156b9cb3eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:31.149099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4f3bb-3fa2-4b6d-9156-1759981f3849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:31.149962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4f3bb-3fa2-4b6d-9156-1759981f3849 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:31.150437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4f3bb-3fa2-4b6d-9156-1759981f3849 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:31.151069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4f3bb-3fa2-4b6d-9156-1759981f3849 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:31.170217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-93856ca8-0387-458b-8018-0156b9cb3eb9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:31.171466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-93856ca8-0387-458b-8018-0156b9cb3eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:31.239061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4f3bb-3fa2-4b6d-9156-1759981f3849 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:31.241004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4f3bb-3fa2-4b6d-9156-1759981f3849 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:31.249238 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 429/866] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:31.310235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:31.354223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 438/867] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:31 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:30:31.359967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93856ca8-0387-458b-8018-0156b9cb3eb9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:31.458661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:31.471190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93856ca8-0387-458b-8018-0156b9cb3eb9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:31.509870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:31.510833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:31.511262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:31.511688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:31.522657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:31.524275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:31.540111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:31.541349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:31.542477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:31.542477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:31.542477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:31.544116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 430/868] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:31.550067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4f3bb-3fa2-4b6d-9156-1759981f3849 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:31.554453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:31.680382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c730bb6-2bea-4621-866a-abd764a16cb8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:31.681396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4f3bb-3fa2-4b6d-9156-1759981f3849 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:31.711312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:31.716834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a988c79-27a9-4d20-a844-2ac8068b37f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:31.717736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988c79-27a9-4d20-a844-2ac8068b37f8 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:31.717878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988c79-27a9-4d20-a844-2ac8068b37f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:31.718080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988c79-27a9-4d20-a844-2ac8068b37f8 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:31.718745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a988c79-27a9-4d20-a844-2ac8068b37f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:31.720290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a988c79-27a9-4d20-a844-2ac8068b37f8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:31.723436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:31.727697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:31.727697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:31.727697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:31.727697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:31.727697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:31.731286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:31.747229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 439/869] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:31.752964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:31.774755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:31.809593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 431/870] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:30:31 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:31.882038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be119e0-e4f7-4145-a5fe-78cfc171ffe0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:31.929872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:31.930535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:31.930819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:31.931093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:31.939426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:31.940899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:31.955245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:31.956198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:31.957087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 440/871] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:31.968972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a988c79-27a9-4d20-a844-2ac8068b37f8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:32.102603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a988c79-27a9-4d20-a844-2ac8068b37f8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:32.142355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510890d8-77a5-4253-8c97-e45bd6220ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:32.142968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510890d8-77a5-4253-8c97-e45bd6220ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:32.143244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510890d8-77a5-4253-8c97-e45bd6220ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:32.143515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510890d8-77a5-4253-8c97-e45bd6220ba9 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:32.144251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510890d8-77a5-4253-8c97-e45bd6220ba9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:32.144553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510890d8-77a5-4253-8c97-e45bd6220ba9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:32.146944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510890d8-77a5-4253-8c97-e45bd6220ba9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:32.160206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 432/872] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:32.166158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:32.250476 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:32.278601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539f69-d6ab-46ae-b407-64aaea58788c None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:32.297992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f339552-ee48-4d2c-ae7f-63e820708e46 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:32.298684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f339552-ee48-4d2c-ae7f-63e820708e46 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:32.298920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f339552-ee48-4d2c-ae7f-63e820708e46 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:32.300394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 433/873] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:32.308406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f36e-450c-4815-bfe3-53b5703e6478 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:32.309011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f36e-450c-4815-bfe3-53b5703e6478 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:32.309518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f36e-450c-4815-bfe3-53b5703e6478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:32.316734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdcd769-e8d1-4cf5-a480-aefd04d6aada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:32.317928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcd769-e8d1-4cf5-a480-aefd04d6aada None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:32.318203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcd769-e8d1-4cf5-a480-aefd04d6aada None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:32.318483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcd769-e8d1-4cf5-a480-aefd04d6aada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:32.319223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdcd769-e8d1-4cf5-a480-aefd04d6aada None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=364e084e459d4263815ab59720d039e4) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:32.319533 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdcd769-e8d1-4cf5-a480-aefd04d6aada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:32.320981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdcd769-e8d1-4cf5-a480-aefd04d6aada None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:32.327630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 441/874] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:32 2022] GET /identity/v3/role_assignments?user.id=364e084e459d4263815ab59720d039e4&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:32.331735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-3ca4f36e-450c-4815-bfe3-53b5703e6478 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:32.334359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3ca4f36e-450c-4815-bfe3-53b5703e6478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:32.379334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe3fd-7497-40b2-ae12-f2039a4d89aa None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:32.380214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe3fd-7497-40b2-ae12-f2039a4d89aa None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:32.380540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe3fd-7497-40b2-ae12-f2039a4d89aa None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:32.383056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 442/875] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:32.390617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05feb9c-68cd-4dbb-8bb1-2eacefcb8547 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:32.390951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05feb9c-68cd-4dbb-8bb1-2eacefcb8547 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:32.391152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05feb9c-68cd-4dbb-8bb1-2eacefcb8547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:32.399320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:32.413833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-e05feb9c-68cd-4dbb-8bb1-2eacefcb8547 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:32.415414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e05feb9c-68cd-4dbb-8bb1-2eacefcb8547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:32.432084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 434/876] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:32.507458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 443/877] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:32.707320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4f36e-450c-4815-bfe3-53b5703e6478 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:32.793620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05feb9c-68cd-4dbb-8bb1-2eacefcb8547 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:32.822409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4f36e-450c-4815-bfe3-53b5703e6478 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:32.868403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37ab32-08ba-4006-85ee-e36ee42be08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:32.869392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37ab32-08ba-4006-85ee-e36ee42be08b None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:32.869919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37ab32-08ba-4006-85ee-e36ee42be08b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:32.870414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37ab32-08ba-4006-85ee-e36ee42be08b None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:32.871475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37ab32-08ba-4006-85ee-e36ee42be08b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:32.872007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:32.873908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37ab32-08ba-4006-85ee-e36ee42be08b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:32.912807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05feb9c-68cd-4dbb-8bb1-2eacefcb8547 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:32.955443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 435/878] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:30:32 2022] POST /identity/v3/services => generated 269 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:32.979111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:32.979946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:32.980324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:32.980694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:32.981395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:32.983000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:32.994603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:32.995724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:32.996583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:32.999646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 444/879] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:33.010020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37ab32-08ba-4006-85ee-e36ee42be08b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:33.069579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:33.139873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37ab32-08ba-4006-85ee-e36ee42be08b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:33.178399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34dbc7f-9648-4641-8c29-893107fb25aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:33.179010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dbc7f-9648-4641-8c29-893107fb25aa None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:33.179294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dbc7f-9648-4641-8c29-893107fb25aa None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:33.179567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dbc7f-9648-4641-8c29-893107fb25aa None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:33.180210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34dbc7f-9648-4641-8c29-893107fb25aa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:33.180509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34dbc7f-9648-4641-8c29-893107fb25aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:33.181883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34dbc7f-9648-4641-8c29-893107fb25aa None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:33.187235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 436/880] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:33.193488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:33.313407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d5313-80d3-41d0-a648-82f64b3b9dec None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:33.356178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:33.361889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec82350-1ef6-497d-87f5-2f988c2e3854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:33.362088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82350-1ef6-497d-87f5-2f988c2e3854 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:33.362311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82350-1ef6-497d-87f5-2f988c2e3854 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:33.365884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82350-1ef6-497d-87f5-2f988c2e3854 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:33.365884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82350-1ef6-497d-87f5-2f988c2e3854 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:33.365884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82350-1ef6-497d-87f5-2f988c2e3854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:33.365884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82350-1ef6-497d-87f5-2f988c2e3854 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:33.371735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 445/881] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:33.956901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1b676-8327-4957-81c3-c121593a5785 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:33.957187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1b676-8327-4957-81c3-c121593a5785 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:33.958044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1b676-8327-4957-81c3-c121593a5785 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:33.959702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 437/882] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:33.965431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82702-3faa-4f73-bbea-9f6fe2a29781 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:33.965739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82702-3faa-4f73-bbea-9f6fe2a29781 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:33.966022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82702-3faa-4f73-bbea-9f6fe2a29781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:33.991291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d2a82702-3faa-4f73-bbea-9f6fe2a29781 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:33.998005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d2a82702-3faa-4f73-bbea-9f6fe2a29781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:34.075802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 446/883] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:34.428591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1b676-8327-4957-81c3-c121593a5785 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:34.513466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8994a-05f1-48f8-9fd3-84c2cf2160a2 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:34.514043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8994a-05f1-48f8-9fd3-84c2cf2160a2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:34.514043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8994a-05f1-48f8-9fd3-84c2cf2160a2 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:34.515721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 447/884] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:34.525886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09b8d-4db8-4725-875f-6eceec793265 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:34.525886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09b8d-4db8-4725-875f-6eceec793265 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:34.525886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09b8d-4db8-4725-875f-6eceec793265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:34.553879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-35d09b8d-4db8-4725-875f-6eceec793265 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:34.557881 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1b676-8327-4957-81c3-c121593a5785 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:34.558086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-35d09b8d-4db8-4725-875f-6eceec793265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:34.558314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:34.601177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:34.603631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:34.603631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:34.603631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:34.604075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:34.605421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:34.617145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:34.618263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:34.619115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 438/885] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:34.625836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:34.716275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 448/886] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:34.762129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa4862-4f87-4132-92ff-fffceb52040d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:34.811726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:34.812361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:34.813476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:34.813476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:34.813806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:34.814163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:34.815557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:34.826519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 439/887] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:34.832690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d09b8d-4db8-4725-875f-6eceec793265 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:34.919932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:34.969300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d09b8d-4db8-4725-875f-6eceec793265 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:35.041189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:35.042276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.042852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.043327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.044176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:35.044796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:35.046645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:35.047806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:35.054047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 449/888] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:35.061829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:35.116535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:35.202857 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5443e1-aa78-4a7a-b422-dc5a1d400d00 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:35.224591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:35.252095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a7b19-fc5e-433a-9445-c06b2d2d664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:35.253177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7b19-fc5e-433a-9445-c06b2d2d664c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.253804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7b19-fc5e-433a-9445-c06b2d2d664c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.254915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7b19-fc5e-433a-9445-c06b2d2d664c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.255856 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7b19-fc5e-433a-9445-c06b2d2d664c None admin] RBAC: Adding query filter params (interface=public, service_id=758cd1e3c6f04650acdf5ddbb0b0f455, region_id=RegionOne) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:35.257798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7b19-fc5e-433a-9445-c06b2d2d664c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:35.259484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7b19-fc5e-433a-9445-c06b2d2d664c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:35.266481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 450/889] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 17:30:35 2022] GET /identity/v3/endpoints?service_id=758cd1e3c6f04650acdf5ddbb0b0f455&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:35.268087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433c615-6bb6-4fac-ae9e-7defe68573a1 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.268594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433c615-6bb6-4fac-ae9e-7defe68573a1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.269086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433c615-6bb6-4fac-ae9e-7defe68573a1 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.271695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 451/890] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:35.272694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433c615-6bb6-4fac-ae9e-7defe68573a1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:35.287354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb45617-821b-4e94-a42e-e2a6c4ddeb78 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:35.339926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b79e70-c023-46c7-a856-46f3ad6efdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:35.340607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b79e70-c023-46c7-a856-46f3ad6efdda None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.340888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b79e70-c023-46c7-a856-46f3ad6efdda None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.341160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b79e70-c023-46c7-a856-46f3ad6efdda None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.360096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b79e70-c023-46c7-a856-46f3ad6efdda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-58b79e70-c023-46c7-a856-46f3ad6efdda None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:35.369438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:35.371354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:35.372713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 440/891] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 17:30:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:35.373456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aceb5-b192-4007-95d2-554cd228f440 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.373456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aceb5-b192-4007-95d2-554cd228f440 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.373456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aceb5-b192-4007-95d2-554cd228f440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.396025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-872aceb5-b192-4007-95d2-554cd228f440 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:35.397798 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-872aceb5-b192-4007-95d2-554cd228f440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:35.429094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433c615-6bb6-4fac-ae9e-7defe68573a1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:35.486183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09e777-8b63-4b26-84ce-201bc193debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:35.486918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09e777-8b63-4b26-84ce-201bc193debc None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.487560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09e777-8b63-4b26-84ce-201bc193debc None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.487845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09e777-8b63-4b26-84ce-201bc193debc None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.489568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09e777-8b63-4b26-84ce-201bc193debc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:35.491033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09e777-8b63-4b26-84ce-201bc193debc None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:35.505992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 441/892] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:35.505992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872aceb5-b192-4007-95d2-554cd228f440 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:35.506635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 452/893] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:30:35 2022] GET /identity/v3/services => generated 1268 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:35.537815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:35.592627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a6f7e-4b66-4f99-8d21-98e82b312ccf None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.593169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a6f7e-4b66-4f99-8d21-98e82b312ccf None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.593169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a6f7e-4b66-4f99-8d21-98e82b312ccf None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.597056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 453/894] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:35.604134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0d84c-a669-4180-b652-3c8a80733200 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.604583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0d84c-a669-4180-b652-3c8a80733200 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.604878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0d84c-a669-4180-b652-3c8a80733200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.646653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-55f0d84c-a669-4180-b652-3c8a80733200 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:35.649728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-55f0d84c-a669-4180-b652-3c8a80733200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:35.651988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872aceb5-b192-4007-95d2-554cd228f440 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:35.706816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:35.707475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.707771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.710084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:35.710084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:35.710084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:35.711224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:35.730152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 442/895] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 17:30:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:35.732772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:35.752451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:35.823870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 454/896] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:35.836285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0d84c-a669-4180-b652-3c8a80733200 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:35.913784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:35.939242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316c0d5-b5de-4518-aeea-317174df3e2f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:35.993599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:35.994275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:35.994560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:35.994836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:36.012395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0d84c-a669-4180-b652-3c8a80733200 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:36.018178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:36.020528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:36.044582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:36.045608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:36.046552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:36.046552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:36.046552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:36.050981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 443/897] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:36.052498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:36.068562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:36.069790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:36.070331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:36.070800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:36.071601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:36.076086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:36.087482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:36.088501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:36.089380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:36.092303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 455/898] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 24 17:30:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:36.099335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:36.158153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:36.189086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed015d6-e6ad-436c-ac48-453f0953b7fb None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:36.237524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:36.238395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:36.239065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:36.239813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:36.239982 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:36.240475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:36.241845 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:36.244525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7113d-a2ea-4bdf-baa6-10d85253eed9 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:36.244896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:36.265774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 444/899] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:36.265774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:36.291289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:36.292200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:36.292551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:36.292888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:36.293629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:36.294020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:36.295552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:36.306042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 456/900] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:36.307796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:36.403562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789072b-f411-4407-b988-b355f5f0b117 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:36.442816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25ed65-8bfb-4096-9e75-b42984d9dcc1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:36.444555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66e58-85bd-44fb-b35f-729b22fc5cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:36.445454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:36.445947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66e58-85bd-44fb-b35f-729b22fc5cde None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:36.447498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66e58-85bd-44fb-b35f-729b22fc5cde None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:36.448109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66e58-85bd-44fb-b35f-729b22fc5cde None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:36.448977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc66e58-85bd-44fb-b35f-729b22fc5cde None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:36.450613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc66e58-85bd-44fb-b35f-729b22fc5cde None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:36.485063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:36.485795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:36.486482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:36.486967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:36.497344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:36.499040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:36.511646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:36.512674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 457/901] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:30:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:36.513661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:36.563236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 445/902] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:30:36 2022] POST /identity/v3/services => generated 241 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:36.567237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66e58-85bd-44fb-b35f-729b22fc5cde None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:36.698164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed4061-ac37-4605-b2d9-9664fae296d5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:36.718768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:36.780336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:36.781145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:36.781425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:36.781696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:36.782600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:36.783522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:36.784944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:36.794937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 458/903] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:36.798562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66e58-85bd-44fb-b35f-729b22fc5cde None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:36.800191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:36.842648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:36.885096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:36.886545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:36.887115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:36.887853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:36.891188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:36.891765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:36.894683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:36.943033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a762c6-20a0-42b2-80c6-c8b1ef91cd5c None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:36.947380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 446/904] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 17:30:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:36.955033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:37.029885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:37.031551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:37.032148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:37.034552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:37.036493 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] RBAC: Adding query filter params (interface=public, service_id=5e0cce297cf847778698029c16af4316, region_id=RegionOne) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:37.037679 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:37.039480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:37.054461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 459/905] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 17:30:36 2022] GET /identity/v3/endpoints?service_id=5e0cce297cf847778698029c16af4316&interface=public®ion_id=RegionOne => generated 194 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:37.063271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:37.177843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b961ee-65cc-41ba-9062-24da157848c4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:37.222382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:37.250574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:37.260500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b1499-9936-4693-911b-e8fbad140ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:37.261478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1499-9936-4693-911b-e8fbad140ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:37.262013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1499-9936-4693-911b-e8fbad140ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:37.262514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1499-9936-4693-911b-e8fbad140ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:37.280141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07080fb-6f7c-4e19-b0c5-2830cb42a1c2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:37.346798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26eb003-895c-4303-87cf-1cd213ea75e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:37.347421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26eb003-895c-4303-87cf-1cd213ea75e0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:37.347746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26eb003-895c-4303-87cf-1cd213ea75e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:37.348464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26eb003-895c-4303-87cf-1cd213ea75e0 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:37.349635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26eb003-895c-4303-87cf-1cd213ea75e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:37.351251 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26eb003-895c-4303-87cf-1cd213ea75e0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:37.365338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b1499-9936-4693-911b-e8fbad140ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:37.366637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 460/906] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:30:37 2022] GET /identity/v3/services => generated 1497 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:37.367629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b1499-9936-4693-911b-e8fbad140ff3 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:37.494946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 447/907] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:36 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:30:37.502936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26eb003-895c-4303-87cf-1cd213ea75e0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:37.608444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ed4fc-f839-4391-92f2-a9b362a1a54a None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:37.608999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ed4fc-f839-4391-92f2-a9b362a1a54a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:37.608999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ed4fc-f839-4391-92f2-a9b362a1a54a None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:37.610461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 448/908] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:37.616850 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fd51-304d-4b3b-9d31-411b2763905d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:37.617123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fd51-304d-4b3b-9d31-411b2763905d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:37.617397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fd51-304d-4b3b-9d31-411b2763905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:37.636614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26eb003-895c-4303-87cf-1cd213ea75e0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:37.643561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ccc3fd51-304d-4b3b-9d31-411b2763905d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:37.647111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ccc3fd51-304d-4b3b-9d31-411b2763905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:37.662671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:37.690044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:37.690848 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:37.691235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:37.691630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:37.707243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:37.708728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:37.734087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:37.735119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:37.736083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:37.736083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:37.736083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:37.736083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 461/909] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:37.745132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:37.757328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:37.774582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 449/910] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:37.880110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489475a-d1fe-448b-8bb2-7f558fa848ac None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:37.930329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:37.930956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:37.931242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:37.931514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:37.932178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:37.932483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:37.934089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:37.948994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 462/911] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:37.960931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3fd51-304d-4b3b-9d31-411b2763905d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:38.096951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:38.115195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3fd51-304d-4b3b-9d31-411b2763905d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:38.156759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:38.157796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:38.158330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:38.158972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:38.169079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:38.170995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:38.185593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:38.186856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:38.188048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:38.188048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:38.188048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:38.188048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:38.188048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:38.188048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:38.188048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:38.188048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:38.196060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 450/912] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:38.197729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:38.201824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:38.219045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:38.298996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923eda6c-1cc5-4a12-9d9a-862903067af1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:38.328820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7ea8e-ea40-4cd9-83ea-9335fac19044 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:38.357177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:38.357763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:38.358129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:38.358406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:38.359006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:38.360393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:38.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:38.375923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:30:38.376771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:38.378356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 463/913] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:38.386931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:38.399975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:38.400658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:38.400939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:38.401212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:38.402021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:38.402329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:38.403783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:38.436378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 451/914] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:38.443611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:38.590742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd080ffe-6125-47d0-b175-42d913ea7e33 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:38.610320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:38.627042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089861d-0592-4198-9a08-9ee7e1e36f65 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:38.645009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:38.665590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:38.667189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:38.667648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:38.668094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:38.669049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:38.670695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:38.672259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:38.687063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 464/915] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:38.689277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:38.694771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3c8bc-f6bf-4f74-a9aa-2c5bd15db257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:38.695807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3c8bc-f6bf-4f74-a9aa-2c5bd15db257 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:38.697578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3c8bc-f6bf-4f74-a9aa-2c5bd15db257 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:38.697578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3c8bc-f6bf-4f74-a9aa-2c5bd15db257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:38.698184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3c8bc-f6bf-4f74-a9aa-2c5bd15db257 None admin] RBAC: Adding query filter params (role.id=0a24fcf372404d29a48ce7ff77aac174, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=364e084e459d4263815ab59720d039e4) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:38.698719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3c8bc-f6bf-4f74-a9aa-2c5bd15db257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:38.706108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3c8bc-f6bf-4f74-a9aa-2c5bd15db257 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:38.715416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 452/916] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:38 2022] GET /identity/v3/role_assignments?user.id=364e084e459d4263815ab59720d039e4&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=0a24fcf372404d29a48ce7ff77aac174 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:38.759311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4b5c-ccc3-4d4f-988f-99b7f310ec24 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:38.759609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4b5c-ccc3-4d4f-988f-99b7f310ec24 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:38.759892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4b5c-ccc3-4d4f-988f-99b7f310ec24 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:38.761634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 453/917] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:38.767891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a9981-8c5e-4f03-8bee-39ec0cf99482 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:38.768194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a9981-8c5e-4f03-8bee-39ec0cf99482 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:38.768493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a9981-8c5e-4f03-8bee-39ec0cf99482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:38.792357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-bc5a9981-8c5e-4f03-8bee-39ec0cf99482 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:38.794563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bc5a9981-8c5e-4f03-8bee-39ec0cf99482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:38.873055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3222e-654c-42b1-ac50-d9e7a856716d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:38.939570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:38.939570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:38.939570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:38.939570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:38.940032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:38.940032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:38.941984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 454/918] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:38.942101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:38.953253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 465/919] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 17:30:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:38.961909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a9981-8c5e-4f03-8bee-39ec0cf99482 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:39.032949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:39.086051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a9981-8c5e-4f03-8bee-39ec0cf99482 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:39.132857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa2085-be9f-41b6-a9e3-02e12a3a0aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:39.133452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa2085-be9f-41b6-a9e3-02e12a3a0aa0 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:39.133731 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa2085-be9f-41b6-a9e3-02e12a3a0aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:39.134044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa2085-be9f-41b6-a9e3-02e12a3a0aa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:39.134719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa2085-be9f-41b6-a9e3-02e12a3a0aa0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:39.136172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa2085-be9f-41b6-a9e3-02e12a3a0aa0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:39.253386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 455/920] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:39.278720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:39.350594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:39.406863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6acba3-b043-4e94-8029-2cbf3364e9b2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:39.470569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d152c1-0f44-4db7-9b09-de66d330f97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:39.471637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d152c1-0f44-4db7-9b09-de66d330f97f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:39.472104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d152c1-0f44-4db7-9b09-de66d330f97f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:39.472550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d152c1-0f44-4db7-9b09-de66d330f97f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:39.473989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d152c1-0f44-4db7-9b09-de66d330f97f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:39.475970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d152c1-0f44-4db7-9b09-de66d330f97f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-89d152c1-0f44-4db7-9b09-de66d330f97f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:39.493561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:39.494998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:39.496147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:39.496991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 466/921] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:39.504642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa2085-be9f-41b6-a9e3-02e12a3a0aa0 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:39.639469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa2085-be9f-41b6-a9e3-02e12a3a0aa0 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:39.657555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:39.665235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13804fe-bbba-43c7-b66c-93267e4eff38 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:39.665748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13804fe-bbba-43c7-b66c-93267e4eff38 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:39.666149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13804fe-bbba-43c7-b66c-93267e4eff38 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:39.667643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 467/922] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:39.675109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca56e9-6fff-4ae8-a6c1-87815e439af3 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:39.675519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca56e9-6fff-4ae8-a6c1-87815e439af3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:39.675951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca56e9-6fff-4ae8-a6c1-87815e439af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:39.694589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:39.695518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:39.695953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:39.696368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:39.697283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:39.697904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:39.701010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:39.706757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-48ca56e9-6fff-4ae8-a6c1-87815e439af3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:39.707632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 456/923] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:39.711625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-48ca56e9-6fff-4ae8-a6c1-87815e439af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:39.717774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:39.852102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 468/924] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 24 17:30:39.910244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615a6c4-ca3b-4794-92d0-99184d502fbf None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:39.952639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:39.953534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:39.954009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:39.954435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:39.955401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:39.955907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:39.958174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:39.967090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 457/925] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:39.981981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca56e9-6fff-4ae8-a6c1-87815e439af3 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:40.053391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:40.118878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca56e9-6fff-4ae8-a6c1-87815e439af3 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:40.154935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:40.155721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:40.156078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:40.156450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:40.157265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] RBAC: Adding query filter params (interface=public, service_id=7d333d58d51c47a88fb5b876ac311c3a, region_id=RegionOne) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:40.157668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:40.159187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:40.167265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 469/926] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 17:30:39 2022] GET /identity/v3/endpoints?service_id=7d333d58d51c47a88fb5b876ac311c3a&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:40.173147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:40.174044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:40.266585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:40.294926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed691f3-d119-4aa6-83d2-2269b49fceb2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:40.305997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:40.341239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:40.342319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:40.343927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:40.345378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:40.346360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:40.349259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:40.363636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:40.364846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:30:40.365974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:40.372993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 470/927] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 24 17:30:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:40.385882 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:40.393016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de59afe-4fe7-45ae-8b66-b59f6837ae42 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:40.473888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a8bcb-9254-4224-941f-02a75a287a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:40.473888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a8bcb-9254-4224-941f-02a75a287a14 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:40.473888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a8bcb-9254-4224-941f-02a75a287a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:40.473888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a8bcb-9254-4224-941f-02a75a287a14 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:40.474304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a8bcb-9254-4224-941f-02a75a287a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:40.474304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a8bcb-9254-4224-941f-02a75a287a14 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:40.505787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 458/928] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:30:40 2022] GET /identity/v3/services => generated 1497 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:40.611836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900884f4-fe88-4e76-a31a-54d7842fc106 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:40.661401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:40.662144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:40.662425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:40.662697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:40.663402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:40.663742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:40.665165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:40.671874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 471/929] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:40.678767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a8bcb-9254-4224-941f-02a75a287a14 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:40.809903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a8bcb-9254-4224-941f-02a75a287a14 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:40.829908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:40.850976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18570000-8d8b-4009-b6b3-031a7ee7c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:40.851616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18570000-8d8b-4009-b6b3-031a7ee7c745 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:40.851896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18570000-8d8b-4009-b6b3-031a7ee7c745 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:40.852551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18570000-8d8b-4009-b6b3-031a7ee7c745 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:40.852903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18570000-8d8b-4009-b6b3-031a7ee7c745 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:40.853210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18570000-8d8b-4009-b6b3-031a7ee7c745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:40.854632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18570000-8d8b-4009-b6b3-031a7ee7c745 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:40.860973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 459/930] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:40.870077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:40.915621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:40.949201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738b87d-793c-401a-b070-c380a0c07a1c None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:40.949539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738b87d-793c-401a-b070-c380a0c07a1c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:40.949758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738b87d-793c-401a-b070-c380a0c07a1c None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:40.951411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 460/931] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:40.956686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21193-f0c2-4d8c-8daf-fc17888d3f0c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:40.956872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21193-f0c2-4d8c-8daf-fc17888d3f0c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:40.957675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21193-f0c2-4d8c-8daf-fc17888d3f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:40.999863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9415cdc-5624-4f0f-82b3-5b5b5a683b03 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:41.005265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-68e21193-f0c2-4d8c-8daf-fc17888d3f0c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:41.008137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-68e21193-f0c2-4d8c-8daf-fc17888d3f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:41.043174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735327e9-570b-47a1-8e6f-62081b608576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:41.044018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735327e9-570b-47a1-8e6f-62081b608576 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:41.044310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735327e9-570b-47a1-8e6f-62081b608576 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:41.044585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735327e9-570b-47a1-8e6f-62081b608576 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:41.045262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735327e9-570b-47a1-8e6f-62081b608576 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:41.046755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735327e9-570b-47a1-8e6f-62081b608576 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:41.143741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 461/932] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:41.168134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 472/933] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:41.467240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e21193-f0c2-4d8c-8daf-fc17888d3f0c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:41.586410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:41.590349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e21193-f0c2-4d8c-8daf-fc17888d3f0c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:41.596759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c895b-e08f-4722-b136-59ec05256249 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:41.597025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c895b-e08f-4722-b136-59ec05256249 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:41.597296 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c895b-e08f-4722-b136-59ec05256249 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:41.599718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 473/934] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:41.604968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3dfa5-fc45-4b01-975d-663fd7dd0bb1 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:41.605254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3dfa5-fc45-4b01-975d-663fd7dd0bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:41.605564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3dfa5-fc45-4b01-975d-663fd7dd0bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:41.637544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-35f3dfa5-fc45-4b01-975d-663fd7dd0bb1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:41.637544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-35f3dfa5-fc45-4b01-975d-663fd7dd0bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:41.647644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b4ef-e512-46d8-96b4-895faa9c239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:41.648369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b4ef-e512-46d8-96b4-895faa9c239a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:41.648715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b4ef-e512-46d8-96b4-895faa9c239a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:41.649045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b4ef-e512-46d8-96b4-895faa9c239a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6b4ef-e512-46d8-96b4-895faa9c239a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-7ad6b4ef-e512-46d8-96b4-895faa9c239a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:41.677241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:41.678448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:41.679975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 462/935] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 17:30:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:41.684629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b4ef-e512-46d8-96b4-895faa9c239a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:41.798192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 474/936] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:41.880395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b4ef-e512-46d8-96b4-895faa9c239a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:41.934313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:41.934938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:41.935213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:41.935487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:41.936165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:41.936466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:41.938671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:41.948952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 463/937] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 17:30:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:41.954560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3dfa5-fc45-4b01-975d-663fd7dd0bb1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:41.996070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:42.081289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3dfa5-fc45-4b01-975d-663fd7dd0bb1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:42.119212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:42.119844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.120119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.120401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.131353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:42.132810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:42.136265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:42.150891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:42.151881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:42.152940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:42.152940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:42.152940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:42.152940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 475/938] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:42.165876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:42.185136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:42.296577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:42.340023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab52ef7-9195-4e55-9c11-30493ffe8ea4 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:42.352502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f726ae-abc4-4deb-9056-8d33cc53cd11 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:42.388896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:42.390321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.390809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.391293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.392140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:42.393786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:42.399169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING keystone.server.flask.common [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 17:30:42.418609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:42.419233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.419577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.419947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.421465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:42.421465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:42.422362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:42.442721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 476/939] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:42.444625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:42.464715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.sql.core [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:30:42.464715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 17:30:42.464715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [parameters: {'id': 'b12c4d20e9404983a0da6d266b348e2f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 17:30:42.464715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81887) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b12c4d20e9404983a0da6d266b348e2f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b12c4d20e9404983a0da6d266b348e2f. Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:30:42.486788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result.read() Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 17:30:42.487749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application yield resource Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise self.value Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application session.commit() Aug 24 17:30:42.489285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:30:42.490294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:30:42.491309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result.read() Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:30:42.492210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [parameters: {'id': 'b12c4d20e9404983a0da6d266b348e2f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:42.493258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:42.495657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b12c4d20e9404983a0da6d266b348e2f. Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 464/940] 10.222.0.76 () {60 vars in 1140 bytes} [Wed Aug 24 17:30:42 2022] POST /identity/v3/projects => generated 262 bytes in 353 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:42.501167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:42.626116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514131f-3a99-4ea6-a7f9-7600a6a94671 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:42.632545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:42.662317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef162db-2584-44ed-9c01-213a54e68272 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:42.693688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:42.694236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.694523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.694957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.697132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:42.705133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:42.709357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:42.711078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:42.712551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.712997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.713514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.722413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:42.723668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:42.724588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:42.725700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:42.728782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:42.735481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 477/941] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:30:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:42.736866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:42.748079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:42.749331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:42.750322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:42.750322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:42.750322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:42.750322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:42.750322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:42.750322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:42.750322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:42.750322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:42.752798 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 465/942] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:42.759753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:42.864402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f4004-39b9-480e-87f0-9a0d650e8231 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:42.884865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c98b7-b112-4ca1-9dcc-a6ba0a8d8613 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:42.918931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75a50e-f3b1-423f-b9bc-cca300d3c8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:42.919559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a50e-f3b1-423f-b9bc-cca300d3c8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.919839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a50e-f3b1-423f-b9bc-cca300d3c8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.920109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a50e-f3b1-423f-b9bc-cca300d3c8b2 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.920851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75a50e-f3b1-423f-b9bc-cca300d3c8b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:42.921156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75a50e-f3b1-423f-b9bc-cca300d3c8b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:42.922605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75a50e-f3b1-423f-b9bc-cca300d3c8b2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:42.924760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:42.925556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.926404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.926693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.929053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:42.929374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:42.930871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:42.932948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:42.952607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 478/943] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 17:30:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:42.952607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175ecd-db39-4359-a3ec-0253bea208a3 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.952607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175ecd-db39-4359-a3ec-0253bea208a3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.952607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175ecd-db39-4359-a3ec-0253bea208a3 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.952607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 479/944] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:42.955356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 466/945] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:42.958154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:42.958516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc232d47-da85-4596-96a9-1127082e8fc2 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.958794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc232d47-da85-4596-96a9-1127082e8fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.959133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc232d47-da85-4596-96a9-1127082e8fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.990419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-cc232d47-da85-4596-96a9-1127082e8fc2 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:42.991895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cc232d47-da85-4596-96a9-1127082e8fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:43.069795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:43.097273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12863f0-b8cd-47ef-990b-9bfdd9d78d45 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:43.114568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 480/946] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:43.145575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1750fbf-7512-421e-9855-653c851d5329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:43.146348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1750fbf-7512-421e-9855-653c851d5329 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:43.149966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1750fbf-7512-421e-9855-653c851d5329 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:43.149966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1750fbf-7512-421e-9855-653c851d5329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:43.162514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1750fbf-7512-421e-9855-653c851d5329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:43.164348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1750fbf-7512-421e-9855-653c851d5329 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c1750fbf-7512-421e-9855-653c851d5329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:43.180855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:43.181997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:43.183105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:43.183105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:43.183105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:43.184851 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 467/947] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:43.191167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc232d47-da85-4596-96a9-1127082e8fc2 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:43.322938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc232d47-da85-4596-96a9-1127082e8fc2 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:43.363817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:43.363817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:43.363817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:43.364307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce7f3-34ce-41fd-8e86-a35087fe8a20 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:43.364445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:43.364537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce7f3-34ce-41fd-8e86-a35087fe8a20 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:43.364537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce7f3-34ce-41fd-8e86-a35087fe8a20 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:43.364698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:43.364698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:43.365697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:43.365890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 468/948] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:43.370756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d8c8b-cac6-4de9-99e5-1eedc1b7e670 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:43.371231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d8c8b-cac6-4de9-99e5-1eedc1b7e670 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:43.371653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d8c8b-cac6-4de9-99e5-1eedc1b7e670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:43.379630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 481/949] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:43.386529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:43.412240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-bf1d8c8b-cac6-4de9-99e5-1eedc1b7e670 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:43.413647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bf1d8c8b-cac6-4de9-99e5-1eedc1b7e670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:43.451936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:43.468791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:43.513904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 469/950] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:43.513904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d8c8b-cac6-4de9-99e5-1eedc1b7e670 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:43.528498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f79fd-5d95-4f30-b1f3-d95d3826253b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:43.600990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:43.600990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:43.600990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:43.600990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:43.628648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:43.628648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:43.657878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:43.658974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:43.660054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 482/951] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:43.666093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:43.733880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d8c8b-cac6-4de9-99e5-1eedc1b7e670 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:43.797923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:43.798840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:43.799189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:43.799535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:43.800304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:43.802687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:43.815565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:43.817539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:43.823622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 470/952] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:43.824621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:43.857453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66112d-53e5-4c8a-823b-11042d9a5048 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:43.860141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:43.876373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:43.910876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:43.911753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:43.912377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:43.915813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:43.916880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:43.917130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:43.919629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:43.939888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 483/953] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:43.941082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:43.966687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c1cb6-9eb9-43ab-aa3f-d286c33ebb2a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:44.013377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:44.014050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.014332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.014832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.015430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:44.016903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:44.027484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:44.033801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:44.038571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 471/954] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:44.039411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:44.067789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3a390-dfaf-48d8-96a5-e423df88e8ef None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:44.106185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:44.106957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.107556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.107898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.108717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:44.109094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:44.110739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:44.120683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 484/955] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:44.120683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:44.163923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:44.171561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8d021-2f59-46cd-a20a-2c2d573c2acf None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:44.174201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:44.213403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b675be8-d94a-46fd-9a02-68b86afcf7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:44.214152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b675be8-d94a-46fd-9a02-68b86afcf7b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.214425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b675be8-d94a-46fd-9a02-68b86afcf7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.214704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b675be8-d94a-46fd-9a02-68b86afcf7b8 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/364e084e459d4263815ab59720d039e4/roles/0a24fcf372404d29a48ce7ff77aac174` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.216893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b675be8-d94a-46fd-9a02-68b86afcf7b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=364e084e459d4263815ab59720d039e4, role_id=0a24fcf372404d29a48ce7ff77aac174)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:44.218443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b675be8-d94a-46fd-9a02-68b86afcf7b8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:44.255865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4980b80-f346-4d11-958e-3bcf7e928d96 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:44.294984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:44.295601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.295876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.296157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.296794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] RBAC: Adding query filter params (name=compute) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:44.297104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:44.298534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:44.308681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 485/956] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:44.309330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:44.340165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 472/957] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:44 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/364e084e459d4263815ab59720d039e4/roles/0a24fcf372404d29a48ce7ff77aac174 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:44.341438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b675be8-d94a-46fd-9a02-68b86afcf7b8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:44.471968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:44.486195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76995e-86a8-4ea5-8e12-4c35866eafee None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:44.549044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:44.551300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.551904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.552383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.553261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:44.554592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:44.557400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:44.560533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b675be8-d94a-46fd-9a02-68b86afcf7b8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:44.567730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 486/958] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:44.575222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:44.601629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d31459-6100-4adf-8bfb-df2750d45048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:44.602645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d31459-6100-4adf-8bfb-df2750d45048 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.603145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d31459-6100-4adf-8bfb-df2750d45048 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.603691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d31459-6100-4adf-8bfb-df2750d45048 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.604716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d31459-6100-4adf-8bfb-df2750d45048 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:44.605577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d31459-6100-4adf-8bfb-df2750d45048 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:44.607228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d31459-6100-4adf-8bfb-df2750d45048 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:44.614119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 473/959] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:44.724276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fb2e9-d519-4514-95cc-29cebbf46c3f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:44.765306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741457d-5ca3-4b2f-a7ca-ee55d8967435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:44.766328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741457d-5ca3-4b2f-a7ca-ee55d8967435 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.766816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741457d-5ca3-4b2f-a7ca-ee55d8967435 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.767314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741457d-5ca3-4b2f-a7ca-ee55d8967435 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.768199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741457d-5ca3-4b2f-a7ca-ee55d8967435 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:44.770490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741457d-5ca3-4b2f-a7ca-ee55d8967435 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:44.870450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 487/960] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:45.209399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10486d65-347b-4627-9a91-b7095c812ea3 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:45.209922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10486d65-347b-4627-9a91-b7095c812ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:45.210029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10486d65-347b-4627-9a91-b7095c812ea3 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:45.211716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 474/961] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:45.217226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895ccb6-be50-4cce-bb22-9d865c46597d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:45.217659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895ccb6-be50-4cce-bb22-9d865c46597d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:45.217838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895ccb6-be50-4cce-bb22-9d865c46597d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:45.238096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4895ccb6-be50-4cce-bb22-9d865c46597d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:45.239537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4895ccb6-be50-4cce-bb22-9d865c46597d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:45.242708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:45.318235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 488/962] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:45.634691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10486d65-347b-4627-9a91-b7095c812ea3 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:45.670529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:45.757118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10486d65-347b-4627-9a91-b7095c812ea3 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:45.797816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:45.798461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:45.798743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:45.803585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:45.803585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:45.803585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:45.870378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.sql.core [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:30:45.870378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 17:30:45.870378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [parameters: {'user_id': 'd1dc4a7909a54e33bcf2e8ed51d96342', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 17:30:45.870378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81887) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result.read() Aug 24 17:30:45.877576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:30:45.878481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application yield resource Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise self.value Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application session.commit() Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:30:45.879666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:30:45.880681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:30:45.881590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application result.read() Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:30:45.882526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd1dc4a7909a54e33bcf2e8ed51d96342', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:45.883381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 17:30:45.884202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 17:30:45.885174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 17:30:45.885174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 17:30:45.885174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:30:45.885174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:30:45.885174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 17:30:45.885174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:45.885174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 475/963] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 17:30:45 2022] POST /identity/v3/users => generated 160 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:45.902179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895ccb6-be50-4cce-bb22-9d865c46597d None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:46.013163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895ccb6-be50-4cce-bb22-9d865c46597d None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:46.052440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8d89c-ef78-4cd5-a592-5d77cc44610e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:46.053388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8d89c-ef78-4cd5-a592-5d77cc44610e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:46.053900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8d89c-ef78-4cd5-a592-5d77cc44610e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:46.054410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8d89c-ef78-4cd5-a592-5d77cc44610e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:46.057845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:46.062919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8d89c-ef78-4cd5-a592-5d77cc44610e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:46.064607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8d89c-ef78-4cd5-a592-5d77cc44610e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-3fc8d89c-ef78-4cd5-a592-5d77cc44610e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:46.077321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:46.078390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:30:46.079364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:46.086135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 489/964] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:30:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:46.089007 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:46.211766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:46.223738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0138c-2047-486a-ad38-74e12895baae None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:46.265077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff873d-4d02-45c1-9990-21eaecbdbe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:46.265696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff873d-4d02-45c1-9990-21eaecbdbe6f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:46.266028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff873d-4d02-45c1-9990-21eaecbdbe6f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:46.266330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff873d-4d02-45c1-9990-21eaecbdbe6f None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:46.267046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff873d-4d02-45c1-9990-21eaecbdbe6f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:46.267350 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff873d-4d02-45c1-9990-21eaecbdbe6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:46.269109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff873d-4d02-45c1-9990-21eaecbdbe6f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:46.322470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 476/965] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 17:30:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:46.526005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d321b19-a9a3-482c-a21f-c5ccd52adca4 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:46.526605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d321b19-a9a3-482c-a21f-c5ccd52adca4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:46.526605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d321b19-a9a3-482c-a21f-c5ccd52adca4 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:46.527993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 490/966] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:46.533729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4b74f-c2f0-4010-97bc-3aa6b847ccb7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:46.534049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4b74f-c2f0-4010-97bc-3aa6b847ccb7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:46.534329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4b74f-c2f0-4010-97bc-3aa6b847ccb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:46.556524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-30f4b74f-c2f0-4010-97bc-3aa6b847ccb7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:46.558078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-30f4b74f-c2f0-4010-97bc-3aa6b847ccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:46.647808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 477/967] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:46.904508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6f9f-d65d-4ed1-ac3a-df1fb5b0f218 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:46.909919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6f9f-d65d-4ed1-ac3a-df1fb5b0f218 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:46.909919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6f9f-d65d-4ed1-ac3a-df1fb5b0f218 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:46.909919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 491/968] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:46.912695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07d769-f29d-4041-9e9d-5838e537807c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:46.912695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07d769-f29d-4041-9e9d-5838e537807c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:46.912991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07d769-f29d-4041-9e9d-5838e537807c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:46.933895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9f07d769-f29d-4041-9e9d-5838e537807c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:46.937882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9f07d769-f29d-4041-9e9d-5838e537807c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:46.942545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c6f9f-d65d-4ed1-ac3a-df1fb5b0f218 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:47.025878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 478/969] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:47.094486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd967-c317-458c-b148-dcc14f2c00e9 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.094779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd967-c317-458c-b148-dcc14f2c00e9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.095059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd967-c317-458c-b148-dcc14f2c00e9 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.101561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 479/970] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:47.102926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b9fa-06fc-4560-91f0-3fb647ac217c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.103196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b9fa-06fc-4560-91f0-3fb647ac217c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.103481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b9fa-06fc-4560-91f0-3fb647ac217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.105035 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c6f9f-d65d-4ed1-ac3a-df1fb5b0f218 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:47.132621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ebe9b9fa-06fc-4560-91f0-3fb647ac217c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:47.135496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ebe9b9fa-06fc-4560-91f0-3fb647ac217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:47.136533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:47.142175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70461728-6a44-4c19-8d26-2ee5bda7c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:47.143040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70461728-6a44-4c19-8d26-2ee5bda7c50d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.143503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70461728-6a44-4c19-8d26-2ee5bda7c50d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.143774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70461728-6a44-4c19-8d26-2ee5bda7c50d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.154958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70461728-6a44-4c19-8d26-2ee5bda7c50d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-70461728-6a44-4c19-8d26-2ee5bda7c50d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:47.159588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:47.164024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:47.164932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 492/971] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 17:30:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:47.167027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70461728-6a44-4c19-8d26-2ee5bda7c50d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:47.234431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:47.245899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 480/972] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:47.294281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70461728-6a44-4c19-8d26-2ee5bda7c50d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:47.301279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9b9fa-06fc-4560-91f0-3fb647ac217c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:47.334578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:47.335177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.335462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.335735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.336405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:47.336706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:47.338090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:47.346902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 493/973] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 17:30:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:47.353387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:47.464264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9b9fa-06fc-4560-91f0-3fb647ac217c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:47.488173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fbccc-4807-48b9-95fc-16eb369d0296 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:47.510863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:47.512665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a5166-034d-46f0-b035-9d8fad0a3208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:47.513295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a5166-034d-46f0-b035-9d8fad0a3208 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.513557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a5166-034d-46f0-b035-9d8fad0a3208 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.513831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a5166-034d-46f0-b035-9d8fad0a3208 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.514506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a5166-034d-46f0-b035-9d8fad0a3208 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:47.515889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a5166-034d-46f0-b035-9d8fad0a3208 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:47.530867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:47.531554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.531841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.532118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.543207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:47.544638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:47.559471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:47.560441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:47.561837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:47.561837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:47.561837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:47.561837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 494/974] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:47.561837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:47.585005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:47.591605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 481/975] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:30:47 2022] POST /identity/v3/services => generated 242 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:30:47.594374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a5166-034d-46f0-b035-9d8fad0a3208 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:47.731903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20fbcf-b292-45fa-8dbc-4cb06dc53aa3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:47.787745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a5166-034d-46f0-b035-9d8fad0a3208 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:47.792504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:47.793166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.793453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.793726 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.794457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:47.795839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:47.810711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:47.811776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:47.812874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:47.817413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 495/976] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:47.818497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:47.849282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:47.850303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.850840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.851328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.852236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:47.852738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:47.854400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:47.865722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:47.869667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 482/977] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:47.874910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:47.957406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:47.963963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e16af-5c4a-476c-96d3-b52eb37e805e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.010077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.012474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.013758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.014261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.015104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:48.015604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.015871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a96fc-3621-4727-8334-a0bab684eec2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.017680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.025165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 496/978] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:48.030992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:48.069412 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.070170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.070323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.070604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.078908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.080465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:48.107090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:48.108295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:30:48.109589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:48.113750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 483/979] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:30:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:48.115523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:48.186781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:48.193176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981cfeb-31d6-4236-b451-c596f81ba14e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.241094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2370408-85cb-409c-a565-df886f1cf10f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.255020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.255893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.256348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.256792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.257774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:48.258297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.259761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.263085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:48.266983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 497/980] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:48.273678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:48.295976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.296584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.296866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.297142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.297896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:48.298210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.299627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.325392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 484/981] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:48.330686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:48.468394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac6be4-902f-446b-8b18-65aeec9fcf9a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.523868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.524398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.524668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.524940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.525679 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] RBAC: Adding query filter params (interface=admin, service_id=7d333d58d51c47a88fb5b876ac311c3a, region_id=RegionOne) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:48.526052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.527574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.538695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 498/982] 10.222.0.76 () {56 vars in 1249 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/endpoints?service_id=7d333d58d51c47a88fb5b876ac311c3a&interface=admin®ion_id=RegionOne => generated 193 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:48.541575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfc5a4-fd1f-476b-ae90-72f098999ce7 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.543118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:48.587110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441633f-78fc-4e65-a318-6fc61dc7d92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.587737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441633f-78fc-4e65-a318-6fc61dc7d92d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.588022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441633f-78fc-4e65-a318-6fc61dc7d92d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.588285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441633f-78fc-4e65-a318-6fc61dc7d92d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.592390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:48.665949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:48.689363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af3787-c5b4-4106-97e5-1f7e35893b39 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.725038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441633f-78fc-4e65-a318-6fc61dc7d92d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.726489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441633f-78fc-4e65-a318-6fc61dc7d92d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.753370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e10c-8e63-4cd8-89a0-dfbdef506f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.754971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e10c-8e63-4cd8-89a0-dfbdef506f93 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.755822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e10c-8e63-4cd8-89a0-dfbdef506f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.756373 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e10c-8e63-4cd8-89a0-dfbdef506f93 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.757591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e10c-8e63-4cd8-89a0-dfbdef506f93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.759149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e10c-8e63-4cd8-89a0-dfbdef506f93 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.767349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 499/983] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/services => generated 1727 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:48.887504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 485/984] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:30:48.892655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e10c-8e63-4cd8-89a0-dfbdef506f93 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.024003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:49.034359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e10c-8e63-4cd8-89a0-dfbdef506f93 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.071869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.072503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.072785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.073058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.083779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.085229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:49.100515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:49.101560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:49.103316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:49.103316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:49.103316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:49.103316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 500/985] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:49.107237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441633f-78fc-4e65-a318-6fc61dc7d92d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.184665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:49.216610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441633f-78fc-4e65-a318-6fc61dc7d92d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.256966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.257583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.257943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.258222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.258925 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:49.259231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.260597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.272830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 486/986] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:49.278193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.384562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8da71d-c541-4c00-ad8d-ddf200ffb9eb None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.419260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279dcb8-4a87-4b90-bbfc-740d7aa49766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.419892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279dcb8-4a87-4b90-bbfc-740d7aa49766 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.420171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279dcb8-4a87-4b90-bbfc-740d7aa49766 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.420443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279dcb8-4a87-4b90-bbfc-740d7aa49766 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.428474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279dcb8-4a87-4b90-bbfc-740d7aa49766 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.430038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279dcb8-4a87-4b90-bbfc-740d7aa49766 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-b279dcb8-4a87-4b90-bbfc-740d7aa49766 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:49.442120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:49.443159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:49.444611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:49.444611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:49.444611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:49.444611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:49.444611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:49.444611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:49.444611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:49.444611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:49.445354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 501/987] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:49.448911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.555632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02611f0-59a7-45ef-b560-f4b505fba362 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.583920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb7cb3-9d55-472e-bd2a-0ffcdf4e1ab4 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.584452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb7cb3-9d55-472e-bd2a-0ffcdf4e1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.584452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb7cb3-9d55-472e-bd2a-0ffcdf4e1ab4 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.586029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 502/988] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:49.588745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:49.591141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e769fc-ee98-4c6b-8114-2fe57e804708 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.591420 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e769fc-ee98-4c6b-8114-2fe57e804708 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.591693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e769fc-ee98-4c6b-8114-2fe57e804708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.598905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.604217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.604320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:49.604425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.604425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.604425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:49.604425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.604425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.611866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b4e769fc-ee98-4c6b-8114-2fe57e804708 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:49.613153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b4e769fc-ee98-4c6b-8114-2fe57e804708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:49.616652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 487/989] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:49.622321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.686504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 503/990] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:49.731320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc84c0-e203-4819-8fde-6daccf923c0c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.771615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.772229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.772509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.772784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.773522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] RBAC: Adding query filter params (role.id=0a24fcf372404d29a48ce7ff77aac174, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=364e084e459d4263815ab59720d039e4) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:49.773831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.775362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.786130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 488/991] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/role_assignments?user.id=364e084e459d4263815ab59720d039e4&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=0a24fcf372404d29a48ce7ff77aac174 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:49.958600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e769fc-ee98-4c6b-8114-2fe57e804708 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.070109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e769fc-ee98-4c6b-8114-2fe57e804708 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.105886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:50.113539 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.114692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.115185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.115662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.116481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.118111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:50.128088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:50.129657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:50.130552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:50.137719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 504/992] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:50.139783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.246274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:50.251828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fa640-eb5b-4ef2-a4c3-373fb61d2c6c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.292898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.294436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.294925 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.295394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.296041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] RBAC: Adding query filter params (name=compute) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:50.296351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.297802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.308060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 489/993] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:50.317033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.428461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b92e8-d0f6-4ad1-9338-0a0162bdaa83 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.463501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.464108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.464390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.464669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.465307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:50.465610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.467011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.472125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 505/994] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:50.478295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.590976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf1833-2ebf-48cd-a17b-c8bdaa6f0aac None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.613025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:50.627877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36aedd1-6088-4a9a-9d74-5f38bbe56e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.628567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aedd1-6088-4a9a-9d74-5f38bbe56e0a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.641668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aedd1-6088-4a9a-9d74-5f38bbe56e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.641668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aedd1-6088-4a9a-9d74-5f38bbe56e0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.641668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36aedd1-6088-4a9a-9d74-5f38bbe56e0a None admin] RBAC: Adding query filter params (interface=public, service_id=0a47bb2d6eaa4f08b41fd1bc58fb35a1, region_id=RegionOne) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:50.641668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36aedd1-6088-4a9a-9d74-5f38bbe56e0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.641668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36aedd1-6088-4a9a-9d74-5f38bbe56e0a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.641668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 490/995] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/endpoints?service_id=0a47bb2d6eaa4f08b41fd1bc58fb35a1&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:50.643245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.709210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:50.760080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16566dd-5f81-478b-ae7b-8047e36b87d0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.761210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff264eb9-7bae-4c39-af16-c0303eb753f4 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.761490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff264eb9-7bae-4c39-af16-c0303eb753f4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.761764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff264eb9-7bae-4c39-af16-c0303eb753f4 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.763866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 491/996] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:50.768441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0385-c854-4017-8070-6eb9347cffb7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.769264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0385-c854-4017-8070-6eb9347cffb7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.769264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0385-c854-4017-8070-6eb9347cffb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.796211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-27ea0385-c854-4017-8070-6eb9347cffb7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:50.797405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-27ea0385-c854-4017-8070-6eb9347cffb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:50.817806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78e48-7977-4511-9273-086758692f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.818459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78e48-7977-4511-9273-086758692f85 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.818840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78e48-7977-4511-9273-086758692f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.819014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78e48-7977-4511-9273-086758692f85 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.819632 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d78e48-7977-4511-9273-086758692f85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.821641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d78e48-7977-4511-9273-086758692f85 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.832638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 506/997] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:50.877300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 492/998] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:51.154508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78e48-7977-4511-9273-086758692f85 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:51.260840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78e48-7977-4511-9273-086758692f85 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:51.285340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:51.296345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:51.296976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:51.297249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:51.297574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:51.298174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:51.299597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:51.309668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:51.310676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:30:51.311531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:51.313140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 507/999] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 17:30:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:51.318486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea0385-c854-4017-8070-6eb9347cffb7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:51.425527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea0385-c854-4017-8070-6eb9347cffb7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:51.446119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:51.467456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:51.468072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:51.468461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:51.468700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:51.469295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:51.469599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:51.471002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:51.480546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 493/1000] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:51.482923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:51.591410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c6612-3ab2-4804-8734-ecd1abba1571 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:51.626320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd510e93-b7a7-4a87-af7d-c461d2dfca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:51.627031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510e93-b7a7-4a87-af7d-c461d2dfca3c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:51.627312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510e93-b7a7-4a87-af7d-c461d2dfca3c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:51.627580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510e93-b7a7-4a87-af7d-c461d2dfca3c None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:51.628206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd510e93-b7a7-4a87-af7d-c461d2dfca3c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:51.628538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd510e93-b7a7-4a87-af7d-c461d2dfca3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:51.630336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd510e93-b7a7-4a87-af7d-c461d2dfca3c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:51.635479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 508/1001] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 17:30:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:51.641210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:51.749342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968aa27-1c5a-419e-9226-f3a717157f03 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:51.785059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3943b-ea86-4c52-bff0-9df95ca3d87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:51.785809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3943b-ea86-4c52-bff0-9df95ca3d87a None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:51.786117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3943b-ea86-4c52-bff0-9df95ca3d87a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:51.786391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3943b-ea86-4c52-bff0-9df95ca3d87a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:51.787052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3943b-ea86-4c52-bff0-9df95ca3d87a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:51.788563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3943b-ea86-4c52-bff0-9df95ca3d87a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:51.788964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe7e59-013e-47ce-a91e-f56a829bb48a None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:51.789415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe7e59-013e-47ce-a91e-f56a829bb48a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:51.789509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe7e59-013e-47ce-a91e-f56a829bb48a None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:51.790980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 509/1002] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:51.795866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9536ad-4a26-48a3-b994-c4e29bbd39dd None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:51.796150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9536ad-4a26-48a3-b994-c4e29bbd39dd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:51.796424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9536ad-4a26-48a3-b994-c4e29bbd39dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:51.814789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-fc9536ad-4a26-48a3-b994-c4e29bbd39dd None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:51.816490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-fc9536ad-4a26-48a3-b994-c4e29bbd39dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:51.866518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:51.886740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 510/1003] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:51.905900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 494/1004] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:52.163275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9536ad-4a26-48a3-b994-c4e29bbd39dd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:52.266895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9536ad-4a26-48a3-b994-c4e29bbd39dd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:52.308553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:52.309237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:52.309523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:52.309796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] PATH_INFO: `/v3/services/image` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:52.310408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:52.311764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:52.321220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:52.322551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:52.323578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 511/1005] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 24 17:30:52 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:52.330472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3943b-ea86-4c52-bff0-9df95ca3d87a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:52.373817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:52.442519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3943b-ea86-4c52-bff0-9df95ca3d87a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:52.478724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2235898-5222-462f-831c-ffe98cae7d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:52.479393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2235898-5222-462f-831c-ffe98cae7d59 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:52.479701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2235898-5222-462f-831c-ffe98cae7d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:52.479979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2235898-5222-462f-831c-ffe98cae7d59 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:52.480618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2235898-5222-462f-831c-ffe98cae7d59 None admin] RBAC: Adding query filter params (name=image) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:52.480923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2235898-5222-462f-831c-ffe98cae7d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:52.482346 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2235898-5222-462f-831c-ffe98cae7d59 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:52.492632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 495/1006] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:52.493907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:52.540116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:52.603967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec957e17-757b-4a55-83bd-360a9a1d0095 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:52.638056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671396e-383f-4844-8ee3-ccb78f8e64f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:52.638664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671396e-383f-4844-8ee3-ccb78f8e64f8 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:52.638942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671396e-383f-4844-8ee3-ccb78f8e64f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:52.639213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671396e-383f-4844-8ee3-ccb78f8e64f8 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:52.639843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671396e-383f-4844-8ee3-ccb78f8e64f8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:52.640174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671396e-383f-4844-8ee3-ccb78f8e64f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:52.641646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671396e-383f-4844-8ee3-ccb78f8e64f8 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:52.647178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 512/1007] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:52.822620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160007-d535-4799-a5bf-7fa0bbf4331f None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:52.822970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160007-d535-4799-a5bf-7fa0bbf4331f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:52.823174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160007-d535-4799-a5bf-7fa0bbf4331f None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:52.824686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 496/1008] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:52.829671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e7fea-3f91-41ba-927f-2be47910443b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:52.830134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e7fea-3f91-41ba-927f-2be47910443b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:52.830340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e7fea-3f91-41ba-927f-2be47910443b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:52.849969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d77e7fea-3f91-41ba-927f-2be47910443b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:52.851203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d77e7fea-3f91-41ba-927f-2be47910443b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:52.923903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 513/1009] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:52 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:53.192869 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160007-d535-4799-a5bf-7fa0bbf4331f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:53.299022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160007-d535-4799-a5bf-7fa0bbf4331f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:53.301133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:53.340638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:53.341314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.341583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.341812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.342402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:53.343766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:53.353120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:53.354498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:30:53.355365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:53.357939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 497/1010] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:53.360453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e7fea-3f91-41ba-927f-2be47910443b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:53.397606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:53.466388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e7fea-3f91-41ba-927f-2be47910443b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:53.501877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:53.502785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.502785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.503065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.503873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:53.512545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:53.512545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:53.521791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 514/1011] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:53.523933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:53.632077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3fe8b-070a-4e6d-84b1-faab7e7c9bf5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:53.668822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cfc31-908c-40e8-95f5-d8959a665032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:53.669426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cfc31-908c-40e8-95f5-d8959a665032 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.669707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cfc31-908c-40e8-95f5-d8959a665032 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.670028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cfc31-908c-40e8-95f5-d8959a665032 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.670662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cfc31-908c-40e8-95f5-d8959a665032 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:53.670967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cfc31-908c-40e8-95f5-d8959a665032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:53.672532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cfc31-908c-40e8-95f5-d8959a665032 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:53.682522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 498/1012] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 17:30:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:53.683643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:53.788392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:53.791949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbd0d7-6d34-4cca-be3f-0173201efbc7 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:53.827663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51a9b8-c084-46d6-806e-5e84b857b728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:53.828317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51a9b8-c084-46d6-806e-5e84b857b728 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.828603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51a9b8-c084-46d6-806e-5e84b857b728 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.828877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51a9b8-c084-46d6-806e-5e84b857b728 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.829537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51a9b8-c084-46d6-806e-5e84b857b728 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:53.830933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51a9b8-c084-46d6-806e-5e84b857b728 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:53.876427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98e83d-e46e-48c3-9983-f47645263902 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.876715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98e83d-e46e-48c3-9983-f47645263902 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.876997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98e83d-e46e-48c3-9983-f47645263902 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.879183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 499/1013] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:53.883844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e470b37-cccd-48d0-aeb7-d49718f9f456 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.884128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e470b37-cccd-48d0-aeb7-d49718f9f456 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.884599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e470b37-cccd-48d0-aeb7-d49718f9f456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.903576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2e470b37-cccd-48d0-aeb7-d49718f9f456 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:53.905041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2e470b37-cccd-48d0-aeb7-d49718f9f456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:53.921141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 515/1014] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:53.940102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:54.001964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 500/1015] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:53 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:54.272243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51a9b8-c084-46d6-806e-5e84b857b728 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:54.381558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51a9b8-c084-46d6-806e-5e84b857b728 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:54.416789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d17caa-b2f0-4532-a4b6-6b7907658aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:54.417432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d17caa-b2f0-4532-a4b6-6b7907658aaf None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.417714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d17caa-b2f0-4532-a4b6-6b7907658aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.418015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d17caa-b2f0-4532-a4b6-6b7907658aaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.419187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d17caa-b2f0-4532-a4b6-6b7907658aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:54.420731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d17caa-b2f0-4532-a4b6-6b7907658aaf None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:54.422536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 516/1016] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:54.428433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e470b37-cccd-48d0-aeb7-d49718f9f456 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:54.536399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e470b37-cccd-48d0-aeb7-d49718f9f456 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:54.568625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b768eec-19ac-4cc9-8700-500ea4b50e10 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.568989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b768eec-19ac-4cc9-8700-500ea4b50e10 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.569260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b768eec-19ac-4cc9-8700-500ea4b50e10 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.570895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 517/1017] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:54.572660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3152d-fa50-4120-aed0-c28eaa73a3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:54.573263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3152d-fa50-4120-aed0-c28eaa73a3f1 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.573539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3152d-fa50-4120-aed0-c28eaa73a3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.573806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3152d-fa50-4120-aed0-c28eaa73a3f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.574675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3152d-fa50-4120-aed0-c28eaa73a3f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:54.575915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3152d-fa50-4120-aed0-c28eaa73a3f1 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:54.576057 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac074-8523-44ff-8b12-3d69f147abbb None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.576331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac074-8523-44ff-8b12-3d69f147abbb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.576614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac074-8523-44ff-8b12-3d69f147abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.596123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-174ac074-8523-44ff-8b12-3d69f147abbb None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:54.597359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-174ac074-8523-44ff-8b12-3d69f147abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:54.615767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:54.667829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 518/1018] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:54 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:54.709895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 501/1019] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:30:54 2022] POST /identity/v3/projects => generated 311 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:30:54.951135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ac074-8523-44ff-8b12-3d69f147abbb None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:55.056579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ac074-8523-44ff-8b12-3d69f147abbb None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:55.092081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c5532-4114-42b9-8903-51793408ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:55.092695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c5532-4114-42b9-8903-51793408ac73 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:55.092971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c5532-4114-42b9-8903-51793408ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:55.093237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c5532-4114-42b9-8903-51793408ac73 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:55.093907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c5532-4114-42b9-8903-51793408ac73 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:55.095387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c5532-4114-42b9-8903-51793408ac73 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:55.202975 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 519/1020] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:30:54 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:55.990171 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2df7e-2f65-4a3a-a456-46e407e46309 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:55.990707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2df7e-2f65-4a3a-a456-46e407e46309 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:55.990707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2df7e-2f65-4a3a-a456-46e407e46309 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:55.992220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 502/1021] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:55.997410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442344-9264-4756-9769-b95a2ce60524 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:55.997791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442344-9264-4756-9769-b95a2ce60524 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:55.998026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442344-9264-4756-9769-b95a2ce60524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:56.016895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f1442344-9264-4756-9769-b95a2ce60524 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:56.018165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f1442344-9264-4756-9769-b95a2ce60524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:56.028883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:56.095992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 520/1022] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:56.367965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2df7e-2f65-4a3a-a456-46e407e46309 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:56.380270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:56.481719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2df7e-2f65-4a3a-a456-46e407e46309 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:56.515820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12cb15-2878-40c4-abc3-bb72c718f644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:56.516448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12cb15-2878-40c4-abc3-bb72c718f644 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:56.516725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12cb15-2878-40c4-abc3-bb72c718f644 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:56.516994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12cb15-2878-40c4-abc3-bb72c718f644 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:56.528664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12cb15-2878-40c4-abc3-bb72c718f644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-5d12cb15-2878-40c4-abc3-bb72c718f644 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:56.533317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:56.534303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:30:56.535772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:56.538195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 503/1023] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 17:30:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:56.541089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1442344-9264-4756-9769-b95a2ce60524 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:56.648870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1442344-9264-4756-9769-b95a2ce60524 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:56.667139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166c856-15c1-463f-bc4e-06d6361db58d None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:56.667463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166c856-15c1-463f-bc4e-06d6361db58d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:56.667710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166c856-15c1-463f-bc4e-06d6361db58d None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:56.669188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 504/1024] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:30:56.674672 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd281b-62c3-4fdc-8cd7-38ae40975e85 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:56.674963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd281b-62c3-4fdc-8cd7-38ae40975e85 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:56.675719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd281b-62c3-4fdc-8cd7-38ae40975e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:56.688139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:56.688754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:56.689569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:56.689895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:56.690866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:56.691210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:56.692640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:56.694684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-04cd281b-62c3-4fdc-8cd7-38ae40975e85 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:56.695885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-04cd281b-62c3-4fdc-8cd7-38ae40975e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:56.701697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 521/1025] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 17:30:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:56.707701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:56.710681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:56.771551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:56.790219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 505/1026] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:56.824244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e2e1b-c7cf-4c43-92b6-cf7d46361f7e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:56.860235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:56.860925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:56.861251 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:56.861605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:56.873746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:56.875608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:56.890268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:56.891253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:56.892157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:56.892157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:56.892157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:56.892157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 522/1027] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:56.897267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd281b-62c3-4fdc-8cd7-38ae40975e85 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.006329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd281b-62c3-4fdc-8cd7-38ae40975e85 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.042325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.042938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.043298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.043576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.044281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:57.044593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.046079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.051167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.063505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 506/1028] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:57.064537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.112187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:57.135667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:57.159325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cebf1a-af75-404c-9124-cd4eb354457f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.186523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9920d-1e0e-4281-8ddf-61d5195ee8f7 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.195885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2a63c-d60e-44a9-acb2-07981bffec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.196526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2a63c-d60e-44a9-acb2-07981bffec06 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.196806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2a63c-d60e-44a9-acb2-07981bffec06 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.197076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2a63c-d60e-44a9-acb2-07981bffec06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.198225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2a63c-d60e-44a9-acb2-07981bffec06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.199654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2a63c-d60e-44a9-acb2-07981bffec06 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.201238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 523/1029] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:57.201238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255444e0-f393-4d45-9011-8f25c3cf8b8d None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.201238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255444e0-f393-4d45-9011-8f25c3cf8b8d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.201238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255444e0-f393-4d45-9011-8f25c3cf8b8d None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.201238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 524/1030] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:57.207393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255444e0-f393-4d45-9011-8f25c3cf8b8d None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.224314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87e94b-9edf-4be5-994a-681c251256a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.225190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87e94b-9edf-4be5-994a-681c251256a6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.225468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87e94b-9edf-4be5-994a-681c251256a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.225737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87e94b-9edf-4be5-994a-681c251256a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.233337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87e94b-9edf-4be5-994a-681c251256a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.234804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87e94b-9edf-4be5-994a-681c251256a6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-5c87e94b-9edf-4be5-994a-681c251256a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:57.245816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:57.251812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 507/1031] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:30:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175343-dc48-4437-b08e-e2919338e324 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.253050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175343-dc48-4437-b08e-e2919338e324 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.254054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175343-dc48-4437-b08e-e2919338e324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.266934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-1a175343-dc48-4437-b08e-e2919338e324 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:57.268162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1a175343-dc48-4437-b08e-e2919338e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:57.318543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255444e0-f393-4d45-9011-8f25c3cf8b8d None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.340071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 508/1032] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:57.340920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175343-dc48-4437-b08e-e2919338e324 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.343702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:57.356133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895bf71-ad4a-422f-909a-ccdf1fba76f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.357623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895bf71-ad4a-422f-909a-ccdf1fba76f1 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.357623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895bf71-ad4a-422f-909a-ccdf1fba76f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.357623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895bf71-ad4a-422f-909a-ccdf1fba76f1 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.358297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895bf71-ad4a-422f-909a-ccdf1fba76f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.359779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895bf71-ad4a-422f-909a-ccdf1fba76f1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.399751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:57.451903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 525/1033] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:30:57 2022] POST /identity/v3/users => generated 285 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:57.459109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175343-dc48-4437-b08e-e2919338e324 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.518011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.519641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.520129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.520602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.521596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:57.522217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.523934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.544185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 509/1034] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:30:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:57.551158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895bf71-ad4a-422f-909a-ccdf1fba76f1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.617369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.677368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895bf71-ad4a-422f-909a-ccdf1fba76f1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.723212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a5324-6519-4539-91ae-11e1189a7812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.724012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a5324-6519-4539-91ae-11e1189a7812 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.724435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a5324-6519-4539-91ae-11e1189a7812 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.724833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a5324-6519-4539-91ae-11e1189a7812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.748764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94f1a-8073-4b61-a0e3-7bbe1e84008f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.756987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:57.778837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:57.787513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.788228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.788530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.788783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] PATH_INFO: `/v3/services/image` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.789274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.790673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:57.801416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:57.802669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:30:57.803587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 510/1035] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 24 17:30:57 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:57.808865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.811924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a5324-6519-4539-91ae-11e1189a7812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.813448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a5324-6519-4539-91ae-11e1189a7812 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.944464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a66b6c-3eec-4dfa-8e64-adc54f656119 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.957682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 526/1036] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:57 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:30:57.958746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a5324-6519-4539-91ae-11e1189a7812 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.986828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.987364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.987744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.988053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.988689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] RBAC: Adding query filter params (name=image) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:57.988991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.990464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.996751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 511/1037] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:58.001163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.039170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.051772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.074188 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a5324-6519-4539-91ae-11e1189a7812 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.111327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8dd4c-b2d8-4377-be79-c53f81a54812 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.111526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.111955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.112234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.112579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.123748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.125200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:58.140696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:58.141747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:58.143319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:58.143319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:58.143319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:58.143319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 527/1038] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:30:58.148281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.155332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.155944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.156224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.156494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.157121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] RBAC: Adding query filter params (type=image) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:58.157431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.158825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.169705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 512/1039] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:58.170841 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.257276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8ec81-1848-436a-817d-5fd510ed90ea None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.277137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.281743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab776-17c8-4991-8def-81f3428c5203 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.288793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.295160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.295811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.296091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.296385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.297168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:58.297478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.298861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.318886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 528/1040] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:58.319077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.319077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.331040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.331040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.331284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.331369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] RBAC: Adding query filter params (interface=public, service_id=1548146269174d34aa4bb696e1053c2a, region_id=RegionOne) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:58.331369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.331369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.331369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 513/1041] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/endpoints?service_id=1548146269174d34aa4bb696e1053c2a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:58.333292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.430238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abc21c-a011-4ff2-b3b6-a0c2024e9270 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.445371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0de6c5-4986-4d8d-99e7-25fcc532fd5a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.466728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.467648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.467929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.468207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.475716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.477157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.482373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ce0a8-e11b-4bf3-a4c1-a65acd27bffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.482984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce0a8-e11b-4bf3-a4c1-a65acd27bffd None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.483421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce0a8-e11b-4bf3-a4c1-a65acd27bffd None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.483821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce0a8-e11b-4bf3-a4c1-a65acd27bffd None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.484426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ce0a8-e11b-4bf3-a4c1-a65acd27bffd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.485787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ce0a8-e11b-4bf3-a4c1-a65acd27bffd None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:58.489650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:58.490585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:30:58.491460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 529/1042] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:58.506131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 514/1043] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:58.506339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.557270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.619535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500623c3-427e-41e8-8f59-ba34be2c2537 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.660085 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf247680-d95a-489f-a98c-81f96a279107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.660842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf247680-d95a-489f-a98c-81f96a279107 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.661127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf247680-d95a-489f-a98c-81f96a279107 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.661421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf247680-d95a-489f-a98c-81f96a279107 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.662130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf247680-d95a-489f-a98c-81f96a279107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:58.662437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf247680-d95a-489f-a98c-81f96a279107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.663866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf247680-d95a-489f-a98c-81f96a279107 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.677562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 530/1044] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:58.683506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ce0a8-e11b-4bf3-a4c1-a65acd27bffd None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.713682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.788635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ce0a8-e11b-4bf3-a4c1-a65acd27bffd None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.822331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef228d-0f96-4d75-a980-c81131dd1985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.822851 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef228d-0f96-4d75-a980-c81131dd1985 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.823128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef228d-0f96-4d75-a980-c81131dd1985 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.823395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef228d-0f96-4d75-a980-c81131dd1985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.824184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef228d-0f96-4d75-a980-c81131dd1985 None admin] RBAC: Adding query filter params (role.id=0a24fcf372404d29a48ce7ff77aac174, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=20dd03dd2fa04147bba827f983a58034) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:58.824487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef228d-0f96-4d75-a980-c81131dd1985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.825983 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef228d-0f96-4d75-a980-c81131dd1985 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.832349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 515/1045] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/role_assignments?user.id=20dd03dd2fa04147bba827f983a58034&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=0a24fcf372404d29a48ce7ff77aac174 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:59.494246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7967be5-b6c3-46a3-82cf-a84aadbe7fe1 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.494767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7967be5-b6c3-46a3-82cf-a84aadbe7fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.494767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7967be5-b6c3-46a3-82cf-a84aadbe7fe1 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.496276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 531/1046] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:59.501568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096b25-e40d-44a9-a0dd-87438d57d54e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.502000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096b25-e40d-44a9-a0dd-87438d57d54e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.502179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096b25-e40d-44a9-a0dd-87438d57d54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.521168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ba096b25-e40d-44a9-a0dd-87438d57d54e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:59.522447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ba096b25-e40d-44a9-a0dd-87438d57d54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:59.595228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 516/1047] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:59.865525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7967be5-b6c3-46a3-82cf-a84aadbe7fe1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:59.979299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7967be5-b6c3-46a3-82cf-a84aadbe7fe1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:59.990435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:00.014864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328fe7c-c1f0-41ad-8b5e-15f93dbc5bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.015496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328fe7c-c1f0-41ad-8b5e-15f93dbc5bad None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.015775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328fe7c-c1f0-41ad-8b5e-15f93dbc5bad None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.016049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328fe7c-c1f0-41ad-8b5e-15f93dbc5bad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.027211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328fe7c-c1f0-41ad-8b5e-15f93dbc5bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-3328fe7c-c1f0-41ad-8b5e-15f93dbc5bad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:00.032371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:00.033323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:00.034340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:00.038187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 532/1048] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:30:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:00.039859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba096b25-e40d-44a9-a0dd-87438d57d54e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.064132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:00.151081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba096b25-e40d-44a9-a0dd-87438d57d54e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.187649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.188303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.188586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.188899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.189525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:00.189820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:00.191549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.200763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 517/1049] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:00.206893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328fe7c-c1f0-41ad-8b5e-15f93dbc5bad None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.312153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328fe7c-c1f0-41ad-8b5e-15f93dbc5bad None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.347374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fbb91-8412-454f-931e-37b8a6264b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.348077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbb91-8412-454f-931e-37b8a6264b85 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.348353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbb91-8412-454f-931e-37b8a6264b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.348622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbb91-8412-454f-931e-37b8a6264b85 None admin] PATH_INFO: `/v3/users/181b71f57ab849faa735b9dc479058d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.363218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fbb91-8412-454f-931e-37b8a6264b85 None admin] RBAC: Authorizing `identity:get_user(user_id=181b71f57ab849faa735b9dc479058d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:00.365037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fbb91-8412-454f-931e-37b8a6264b85 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.366975 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 533/1050] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/users/181b71f57ab849faa735b9dc479058d1 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:00.372599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.422698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0f25-016f-4ea0-bbd7-c660f6d77ca5 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.422899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0f25-016f-4ea0-bbd7-c660f6d77ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.423106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0f25-016f-4ea0-bbd7-c660f6d77ca5 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.424591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 534/1051] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:00.430453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f5368-55dc-4f3a-be16-1ffaa767b179 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.430735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f5368-55dc-4f3a-be16-1ffaa767b179 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.431011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f5368-55dc-4f3a-be16-1ffaa767b179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.450448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-237f5368-55dc-4f3a-be16-1ffaa767b179 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:00.451663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-237f5368-55dc-4f3a-be16-1ffaa767b179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:00.462060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:00.477157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:00.481105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12464774-f7b2-426d-b3bd-3e6879f9e4fc None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.516273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69520f-f457-4b93-8265-e652eb67434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.517051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69520f-f457-4b93-8265-e652eb67434a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.517149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69520f-f457-4b93-8265-e652eb67434a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.517399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69520f-f457-4b93-8265-e652eb67434a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.524534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 535/1052] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:31:00.595313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69520f-f457-4b93-8265-e652eb67434a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:00.597410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69520f-f457-4b93-8265-e652eb67434a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.683967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:00.703582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 518/1053] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:00.709627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f5368-55dc-4f3a-be16-1ffaa767b179 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.765151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f51bea-66a4-4d2a-a64a-7e94c1e73f62 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.765483 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f51bea-66a4-4d2a-a64a-7e94c1e73f62 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.765766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f51bea-66a4-4d2a-a64a-7e94c1e73f62 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.767166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 519/1054] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:31:00.772839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08e786-93bf-4e72-af26-c7efc4e36b64 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.773189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08e786-93bf-4e72-af26-c7efc4e36b64 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.773376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08e786-93bf-4e72-af26-c7efc4e36b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.793096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9a08e786-93bf-4e72-af26-c7efc4e36b64 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:00.794466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9a08e786-93bf-4e72-af26-c7efc4e36b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:00.816010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f5368-55dc-4f3a-be16-1ffaa767b179 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.851297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083bddc-d529-414d-9b9f-0b1e7ef17328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.851923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083bddc-d529-414d-9b9f-0b1e7ef17328 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.852199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083bddc-d529-414d-9b9f-0b1e7ef17328 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.852470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083bddc-d529-414d-9b9f-0b1e7ef17328 None admin] PATH_INFO: `/v3/projects/bd926005f58c4ff092bcc2543d0f84f9` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.853602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083bddc-d529-414d-9b9f-0b1e7ef17328 None admin] RBAC: Authorizing `identity:get_project(project_id=bd926005f58c4ff092bcc2543d0f84f9)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:00.855059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083bddc-d529-414d-9b9f-0b1e7ef17328 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.856690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 536/1055] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/projects/bd926005f58c4ff092bcc2543d0f84f9 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:00.862527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083bddc-d529-414d-9b9f-0b1e7ef17328 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.866903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 520/1056] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:31:00.867748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08e786-93bf-4e72-af26-c7efc4e36b64 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.898703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:00.984871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083bddc-d529-414d-9b9f-0b1e7ef17328 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.989096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08e786-93bf-4e72-af26-c7efc4e36b64 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:01.003441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:01.022270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.022998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.023284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.023559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] PATH_INFO: `/v3/services/image` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.024148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:01.025517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:01.027044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.027756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.028140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.028444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.029198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=bd926005f58c4ff092bcc2543d0f84f9, user.id=181b71f57ab849faa735b9dc479058d1) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:01.029511 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:01.030952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:01.035873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:01.041081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:01.041979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 537/1057] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:01.043242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:01.043870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 521/1058] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/role_assignments?user.id=181b71f57ab849faa735b9dc479058d1&scope.project.id=bd926005f58c4ff092bcc2543d0f84f9&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:01.139753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:01.147956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:01.159045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7df43-1c7f-4754-af2f-a7b802eac1fd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:01.199071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.199681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.199971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.200254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.200879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] RBAC: Adding query filter params (name=image) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:01.201181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:01.202600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:01.208828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 538/1059] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:01.214374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:01.266424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f57b-ec69-4d5f-bfbf-1583300529f6 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:01.303675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d5b7-08a6-4e8e-94fd-2833bb3740f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.304331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d5b7-08a6-4e8e-94fd-2833bb3740f2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.304613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d5b7-08a6-4e8e-94fd-2833bb3740f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.304885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d5b7-08a6-4e8e-94fd-2833bb3740f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.317465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4d5b7-08a6-4e8e-94fd-2833bb3740f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c2c4d5b7-08a6-4e8e-94fd-2833bb3740f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:01.322521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:01.324962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:01.325907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 522/1060] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 17:31:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:31:01.329265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e381ff-e397-4970-8018-7ffe7bba8654 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:01.329452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d5b7-08a6-4e8e-94fd-2833bb3740f2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:01.369619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.370263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.370549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.370887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.371530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:01.371837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:01.373098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:01.373218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:01.378791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 539/1061] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:01.384189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:01.405319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:01.444327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d5b7-08a6-4e8e-94fd-2833bb3740f2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:01.483560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.484173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.484471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.484767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.485526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:01.485837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:01.487228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:01.495833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55d17b-c2cd-4c53-9735-0afc69b69bf1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:01.497913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 523/1062] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 17:31:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:01.502290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:01.534708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ea777-3760-4079-8e4e-1a59aa30f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.535401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ea777-3760-4079-8e4e-1a59aa30f71b None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.535730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ea777-3760-4079-8e4e-1a59aa30f71b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.535851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ea777-3760-4079-8e4e-1a59aa30f71b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.536445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ea777-3760-4079-8e4e-1a59aa30f71b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:01.537812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ea777-3760-4079-8e4e-1a59aa30f71b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:01.606256 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:01.610158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab71572-de04-4e49-8672-f8739e2fcc0d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:01.625277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 540/1063] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:31:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:31:01.669009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.670034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.670504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.671005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.686281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:01.688490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:01.705421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:01.706663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:01.707691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:01.707691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:01.707691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:01.713225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 524/1064] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:01.713725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ea777-3760-4079-8e4e-1a59aa30f71b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:01.803109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:01.830745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ea777-3760-4079-8e4e-1a59aa30f71b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:01.866713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:01.867320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:01.867600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:01.867872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:01.868520 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:01.868828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:01.870222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:01.884292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 541/1065] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:01.890134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:02.012020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33f35d-c763-4d2e-9400-039460cb356f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:02.048641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:02.049217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.049498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.049822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.058247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:02.059635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:31:02.071381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:02.072500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:02.073581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 525/1066] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:31:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:02.080187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:02.187325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccdbb0-e7a4-473f-ad9d-24300c616970 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:02.221557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:02.222701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.222983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.223244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.223971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:02.224277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:02.225700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:02.227401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:02.248891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 542/1067] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:31:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:02.250137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:02.259775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:02.354446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903ca79-f506-45e3-9e56-8f24a411ed88 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:02.388066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:02.388699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.388980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.389254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.400595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:02.402060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:02.416769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:02.417865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:02.423878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:02.423878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:02.423878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:02.423878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 526/1068] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:31:02.424138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:02.539435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccc6c3-4178-422b-90d4-a55652e37528 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:02.577663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:02.578311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.578584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.578858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.579511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:02.579819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:02.581306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:02.597896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 543/1069] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:02.599785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:02.662154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:02.707071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108410cc-5738-4d4d-a363-46a872657160 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:02.742829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:02.743459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.743739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.744015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.750868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:02.752320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:31:02.763872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:02.764917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:02.771206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 527/1070] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:02.771742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:02.833743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:02.891626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c64df-52d5-498d-9683-0229fbc84673 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:02.933889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3239a4-cb42-430c-9ce6-74ee77255877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:02.934569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3239a4-cb42-430c-9ce6-74ee77255877 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.934849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3239a4-cb42-430c-9ce6-74ee77255877 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.935121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3239a4-cb42-430c-9ce6-74ee77255877 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.935816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3239a4-cb42-430c-9ce6-74ee77255877 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:02.936124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3239a4-cb42-430c-9ce6-74ee77255877 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:02.937806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3239a4-cb42-430c-9ce6-74ee77255877 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:02.956674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 544/1071] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:02.958100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:02.967614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5145f-1715-45a4-885f-fba1da22d674 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.967900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5145f-1715-45a4-885f-fba1da22d674 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.968177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5145f-1715-45a4-885f-fba1da22d674 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.970209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 545/1072] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:31:02.975445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec5880-012a-4d4b-be47-6fa3b6e1f8cd None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.975739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec5880-012a-4d4b-be47-6fa3b6e1f8cd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.976020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec5880-012a-4d4b-be47-6fa3b6e1f8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.993962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-83ec5880-012a-4d4b-be47-6fa3b6e1f8cd None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:02.995247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-83ec5880-012a-4d4b-be47-6fa3b6e1f8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:03.065144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 546/1073] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:31:03.087537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e0567-6f8b-4789-bab3-0b6b4ddc43cb None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.106308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:03.123899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18340a40-2d8a-4e41-ba02-9c90fec50d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.127114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340a40-2d8a-4e41-ba02-9c90fec50d69 None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.127114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340a40-2d8a-4e41-ba02-9c90fec50d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.127114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340a40-2d8a-4e41-ba02-9c90fec50d69 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/20dd03dd2fa04147bba827f983a58034/roles/0a24fcf372404d29a48ce7ff77aac174` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.130718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18340a40-2d8a-4e41-ba02-9c90fec50d69 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=20dd03dd2fa04147bba827f983a58034, role_id=0a24fcf372404d29a48ce7ff77aac174)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:03.132354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18340a40-2d8a-4e41-ba02-9c90fec50d69 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:03.236764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 528/1074] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:31:02 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/20dd03dd2fa04147bba827f983a58034/roles/0a24fcf372404d29a48ce7ff77aac174 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:03.339487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec5880-012a-4d4b-be47-6fa3b6e1f8cd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.389902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:03.451253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec5880-012a-4d4b-be47-6fa3b6e1f8cd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.487878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf0d8b-d81c-4b8b-8b0d-29f1dff2b9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.488404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0d8b-d81c-4b8b-8b0d-29f1dff2b9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.488685 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0d8b-d81c-4b8b-8b0d-29f1dff2b9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.488979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0d8b-d81c-4b8b-8b0d-29f1dff2b9d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.499500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf0d8b-d81c-4b8b-8b0d-29f1dff2b9d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-fddf0d8b-d81c-4b8b-8b0d-29f1dff2b9d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:03.504193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:03.505199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:03.506573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 547/1075] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:03.511371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18340a40-2d8a-4e41-ba02-9c90fec50d69 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.623445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18340a40-2d8a-4e41-ba02-9c90fec50d69 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.632875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d27787-c549-4e86-88e8-83d85b62a960 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.633414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d27787-c549-4e86-88e8-83d85b62a960 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.633923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d27787-c549-4e86-88e8-83d85b62a960 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.635524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 548/1076] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:03.640406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c99d7-dd22-48ab-8a02-77366b69ddda None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.640697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c99d7-dd22-48ab-8a02-77366b69ddda None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.640981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c99d7-dd22-48ab-8a02-77366b69ddda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.660127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-bb1c99d7-dd22-48ab-8a02-77366b69ddda None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:03.661316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635212d1-2716-495e-8908-b5a236009374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.661468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bb1c99d7-dd22-48ab-8a02-77366b69ddda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:03.661747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635212d1-2716-495e-8908-b5a236009374 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.662049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635212d1-2716-495e-8908-b5a236009374 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.662323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635212d1-2716-495e-8908-b5a236009374 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.662990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635212d1-2716-495e-8908-b5a236009374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:03.663294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635212d1-2716-495e-8908-b5a236009374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:03.664646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635212d1-2716-495e-8908-b5a236009374 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:03.674107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 529/1077] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:03.680836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635212d1-2716-495e-8908-b5a236009374 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.731886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:03.746610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 549/1078] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:31:03.791915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635212d1-2716-495e-8908-b5a236009374 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.828363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0ba4d-1744-4f61-acfe-422f0f49bb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.828994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ba4d-1744-4f61-acfe-422f0f49bb82 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.829277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ba4d-1744-4f61-acfe-422f0f49bb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.829551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ba4d-1744-4f61-acfe-422f0f49bb82 None admin] PATH_INFO: `/v3/users/181b71f57ab849faa735b9dc479058d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.830758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0ba4d-1744-4f61-acfe-422f0f49bb82 None admin] RBAC: Authorizing `identity:get_user(user_id=181b71f57ab849faa735b9dc479058d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:03.832813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0ba4d-1744-4f61-acfe-422f0f49bb82 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:03.834601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 530/1079] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/users/181b71f57ab849faa735b9dc479058d1 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:03.840237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c99d7-dd22-48ab-8a02-77366b69ddda None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.877135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:03.959666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c99d7-dd22-48ab-8a02-77366b69ddda None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.994884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727b346-3134-4ad3-b0f0-02f9a594bf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.995507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727b346-3134-4ad3-b0f0-02f9a594bf45 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.995785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727b346-3134-4ad3-b0f0-02f9a594bf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.996055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727b346-3134-4ad3-b0f0-02f9a594bf45 None admin] PATH_INFO: `/v3/projects/bd926005f58c4ff092bcc2543d0f84f9` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.997228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727b346-3134-4ad3-b0f0-02f9a594bf45 None admin] RBAC: Authorizing `identity:get_project(project_id=bd926005f58c4ff092bcc2543d0f84f9)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:03.998629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727b346-3134-4ad3-b0f0-02f9a594bf45 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.000430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 550/1080] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/projects/bd926005f58c4ff092bcc2543d0f84f9 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:04.006795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0ba4d-1744-4f61-acfe-422f0f49bb82 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.021471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727b346-3134-4ad3-b0f0-02f9a594bf45 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.116463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0ba4d-1744-4f61-acfe-422f0f49bb82 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.126495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:04.127390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:04.131368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727b346-3134-4ad3-b0f0-02f9a594bf45 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.157263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc310b-7305-471d-b462-339a8f9a30dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.158523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc310b-7305-471d-b462-339a8f9a30dd None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.158821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc310b-7305-471d-b462-339a8f9a30dd None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.159398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc310b-7305-471d-b462-339a8f9a30dd None admin] PATH_INFO: `/v3/projects/bd926005f58c4ff092bcc2543d0f84f9/users/181b71f57ab849faa735b9dc479058d1/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.162279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc310b-7305-471d-b462-339a8f9a30dd None admin] RBAC: Authorizing `identity:create_grant(project_id=bd926005f58c4ff092bcc2543d0f84f9, user_id=181b71f57ab849faa735b9dc479058d1, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.163943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc310b-7305-471d-b462-339a8f9a30dd None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.166177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407a899-6d16-4d8e-b71c-5ab417df21f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.166789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407a899-6d16-4d8e-b71c-5ab417df21f3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.170174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407a899-6d16-4d8e-b71c-5ab417df21f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.170174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407a899-6d16-4d8e-b71c-5ab417df21f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.235886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 531/1081] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:31:04 2022] PUT /identity/v3/projects/bd926005f58c4ff092bcc2543d0f84f9/users/181b71f57ab849faa735b9dc479058d1/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:04.250147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407a899-6d16-4d8e-b71c-5ab417df21f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.251581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407a899-6d16-4d8e-b71c-5ab417df21f3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.358979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:04.364556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 551/1082] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:31:04.373532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc310b-7305-471d-b462-339a8f9a30dd None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.500001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc310b-7305-471d-b462-339a8f9a30dd None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.535792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e655b-da8a-45f5-a80e-cb01889df523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.536342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e655b-da8a-45f5-a80e-cb01889df523 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.536641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e655b-da8a-45f5-a80e-cb01889df523 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.536911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e655b-da8a-45f5-a80e-cb01889df523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.547208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e655b-da8a-45f5-a80e-cb01889df523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.548666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e655b-da8a-45f5-a80e-cb01889df523 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-e61e655b-da8a-45f5-a80e-cb01889df523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:04.564298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:04.565419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:04.566420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:04.566420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:04.566420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:04.566420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 532/1083] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:04.572026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407a899-6d16-4d8e-b71c-5ab417df21f3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.679920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407a899-6d16-4d8e-b71c-5ab417df21f3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.715692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa0819-99a6-4838-9139-a02c9441f3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.716311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0819-99a6-4838-9139-a02c9441f3bf None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.716592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0819-99a6-4838-9139-a02c9441f3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.716865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0819-99a6-4838-9139-a02c9441f3bf None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.717517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa0819-99a6-4838-9139-a02c9441f3bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:04.717820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa0819-99a6-4838-9139-a02c9441f3bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.719212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa0819-99a6-4838-9139-a02c9441f3bf None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.738020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 552/1084] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:05.268174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994d18-044b-4480-8ea0-2308a853d1c3 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:05.268767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994d18-044b-4480-8ea0-2308a853d1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:05.268861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994d18-044b-4480-8ea0-2308a853d1c3 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:05.270278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 533/1085] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:05.275614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36c09f-9e37-4d5e-9180-6896869c1cc1 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:05.275907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36c09f-9e37-4d5e-9180-6896869c1cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:05.276184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36c09f-9e37-4d5e-9180-6896869c1cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:05.295056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-eb36c09f-9e37-4d5e-9180-6896869c1cc1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:05.296275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-eb36c09f-9e37-4d5e-9180-6896869c1cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:05.354181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:05.365634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 553/1086] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:05.636475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4994d18-044b-4480-8ea0-2308a853d1c3 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:05.661533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:05.747092 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4994d18-044b-4480-8ea0-2308a853d1c3 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:05.783228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c80c-5077-4b8a-bf19-fc82d5d9b650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:05.783820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c80c-5077-4b8a-bf19-fc82d5d9b650 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:05.784701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c80c-5077-4b8a-bf19-fc82d5d9b650 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:05.785132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c80c-5077-4b8a-bf19-fc82d5d9b650 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:05.795458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84c80c-5077-4b8a-bf19-fc82d5d9b650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-fb84c80c-5077-4b8a-bf19-fc82d5d9b650 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:05.801064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:05.802058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:05.802896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 534/1087] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 17:31:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:05.808221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36c09f-9e37-4d5e-9180-6896869c1cc1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:05.909656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36c09f-9e37-4d5e-9180-6896869c1cc1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:05.945519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:05.946149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:05.946434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:05.946707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:05.947379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:05.947684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:05.949042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:05.957772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 554/1088] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 17:31:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:05.963665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c80c-5077-4b8a-bf19-fc82d5d9b650 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.044901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:06.063965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c80c-5077-4b8a-bf19-fc82d5d9b650 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:06.099487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea16752-d440-4464-ae9a-b78c5eb2e699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:06.100108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea16752-d440-4464-ae9a-b78c5eb2e699 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.100382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea16752-d440-4464-ae9a-b78c5eb2e699 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.100647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea16752-d440-4464-ae9a-b78c5eb2e699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.111160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea16752-d440-4464-ae9a-b78c5eb2e699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:06.113641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea16752-d440-4464-ae9a-b78c5eb2e699 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-6ea16752-d440-4464-ae9a-b78c5eb2e699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:06.127590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:06.129125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:06.130671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:06.130671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:06.130671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:06.130671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 535/1089] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:06.135504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.191105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:06.210699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ad90-bbc2-4369-b113-e914cb20360e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.210992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ad90-bbc2-4369-b113-e914cb20360e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.211273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ad90-bbc2-4369-b113-e914cb20360e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.212785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 536/1090] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:31:06.218304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0150-2b2b-4ae3-aceb-eb54424bda7b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.218612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0150-2b2b-4ae3-aceb-eb54424bda7b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.219014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0150-2b2b-4ae3-aceb-eb54424bda7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.238052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-853c0150-2b2b-4ae3-aceb-eb54424bda7b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:06.239219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863597c3-1ee5-47d9-9426-4689c2e3b29e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:06.239445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-853c0150-2b2b-4ae3-aceb-eb54424bda7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:06.274683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:06.275288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.275562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.275830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.276473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:06.276771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:06.278163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:06.290156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 555/1091] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:06.296919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.312015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 537/1092] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:31:06.398316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2eba6-ae5b-4934-a478-0106cee654ff None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:06.411638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:06.432795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:06.433428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.433712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.434015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.441900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:06.443292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:31:06.455277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:06.456436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:06.457500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 556/1093] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:31:06.462558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c0150-2b2b-4ae3-aceb-eb54424bda7b None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.526144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:06.569190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c0150-2b2b-4ae3-aceb-eb54424bda7b None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:06.587982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.605542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:06.606190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.606472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.606746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.607482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:06.607789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:06.609331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:06.627946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 538/1094] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:06.633795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.695565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0c20-75e4-491f-a59c-4e7713d78746 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:06.730426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0c6d-2b12-4f5c-9d71-d13aa06cc68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:06.731047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0c6d-2b12-4f5c-9d71-d13aa06cc68a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.731324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0c6d-2b12-4f5c-9d71-d13aa06cc68a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.731601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0c6d-2b12-4f5c-9d71-d13aa06cc68a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.737826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11ce38-08fc-435b-b481-75a87e1756c5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:06.742571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0c6d-2b12-4f5c-9d71-d13aa06cc68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-105f0c6d-2b12-4f5c-9d71-d13aa06cc68a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:06.747180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:06.748256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:06.749129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 557/1095] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:31:06.755312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0c6d-2b12-4f5c-9d71-d13aa06cc68a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.759376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:06.775980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432cdec-4b26-44a6-be5a-2df1c71e16f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:06.777655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432cdec-4b26-44a6-be5a-2df1c71e16f2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.777655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432cdec-4b26-44a6-be5a-2df1c71e16f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.777655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432cdec-4b26-44a6-be5a-2df1c71e16f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.786668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:06.848376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432cdec-4b26-44a6-be5a-2df1c71e16f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:06.849829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432cdec-4b26-44a6-be5a-2df1c71e16f2 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:06.856333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0c6d-2b12-4f5c-9d71-d13aa06cc68a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:06.890508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:06.891217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.891507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.891785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.892455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:06.892760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:06.894867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:06.904707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 558/1096] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:06.911276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.958993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 539/1097] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:31:06.960336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432cdec-4b26-44a6-be5a-2df1c71e16f2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.981744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.010216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.016924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426912-d062-4708-947b-cf184878d5a0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.052038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bc39c-64bc-405c-a9f0-cf0aa40f61be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.052670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bc39c-64bc-405c-a9f0-cf0aa40f61be None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.052949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bc39c-64bc-405c-a9f0-cf0aa40f61be None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.053225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bc39c-64bc-405c-a9f0-cf0aa40f61be None admin] PATH_INFO: `/v3/users/181b71f57ab849faa735b9dc479058d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.054398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bc39c-64bc-405c-a9f0-cf0aa40f61be None admin] RBAC: Authorizing `identity:get_user(user_id=181b71f57ab849faa735b9dc479058d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.055879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bc39c-64bc-405c-a9f0-cf0aa40f61be None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.057647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 559/1098] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/users/181b71f57ab849faa735b9dc479058d1 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:07.059915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432cdec-4b26-44a6-be5a-2df1c71e16f2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.064060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bc39c-64bc-405c-a9f0-cf0aa40f61be None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.095345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.095993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.096269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.097241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.107441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.108907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:07.123962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:07.125118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:07.126406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:07.126406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:07.126406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:07.126406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 540/1099] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:31:07.130831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.167881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bc39c-64bc-405c-a9f0-cf0aa40f61be None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.202609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87fee1-b482-4adc-9c5e-c8bf3c25556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.202994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.203313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87fee1-b482-4adc-9c5e-c8bf3c25556f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.203918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87fee1-b482-4adc-9c5e-c8bf3c25556f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.204199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87fee1-b482-4adc-9c5e-c8bf3c25556f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.233943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f200d16-d82f-4786-ab8f-1f9b2f26d760 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.234894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.269504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.270148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.270433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.270712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.271362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:07.271665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.273106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.276340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87fee1-b482-4adc-9c5e-c8bf3c25556f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.277017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87fee1-b482-4adc-9c5e-c8bf3c25556f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.286361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 541/1100] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:07.291906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.379055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 560/1101] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:31:07.385098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87fee1-b482-4adc-9c5e-c8bf3c25556f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.393993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d6de3-c3e4-49b5-b009-329e34761502 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.423211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.428630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.429255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.429535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.429808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.437500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.438974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.448778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:31:07.451338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:07.452357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:07.453265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:31:07.453265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:31:07.453265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:31:07.453265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:31:07.453265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:31:07.453265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:31:07.453265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:07.453265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:07.458229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 542/1102] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:07.464437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.489041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87fee1-b482-4adc-9c5e-c8bf3c25556f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.525839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a54f2-09ee-4ea9-8b4c-5701d3b53969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.526483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a54f2-09ee-4ea9-8b4c-5701d3b53969 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.526759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a54f2-09ee-4ea9-8b4c-5701d3b53969 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.527028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a54f2-09ee-4ea9-8b4c-5701d3b53969 None admin] PATH_INFO: `/v3/projects/bd926005f58c4ff092bcc2543d0f84f9` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.528137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a54f2-09ee-4ea9-8b4c-5701d3b53969 None admin] RBAC: Authorizing `identity:get_project(project_id=bd926005f58c4ff092bcc2543d0f84f9)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.529550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a54f2-09ee-4ea9-8b4c-5701d3b53969 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.531286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 561/1103] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/projects/bd926005f58c4ff092bcc2543d0f84f9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:07.537640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a54f2-09ee-4ea9-8b4c-5701d3b53969 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.561871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febe0db-f4a4-4620-90ea-368a78cbf047 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.599615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.600229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.600515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.600787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.601503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:07.601812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.603287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.617494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 543/1104] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:07.623629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.646967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a54f2-09ee-4ea9-8b4c-5701d3b53969 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.679682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.682477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.688835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6097c-3085-4974-aec3-637147b96e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.689444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6097c-3085-4974-aec3-637147b96e85 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.689726 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6097c-3085-4974-aec3-637147b96e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.690028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6097c-3085-4974-aec3-637147b96e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.691438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6097c-3085-4974-aec3-637147b96e85 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=bd926005f58c4ff092bcc2543d0f84f9, user.id=181b71f57ab849faa735b9dc479058d1) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:07.691748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6097c-3085-4974-aec3-637147b96e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.693172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6097c-3085-4974-aec3-637147b96e85 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.698896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 562/1105] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/role_assignments?user.id=181b71f57ab849faa735b9dc479058d1&scope.project.id=bd926005f58c4ff092bcc2543d0f84f9&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:07.730637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042ae10-4c59-4f8c-bfb8-44e60b6399ca None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.769330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbc3a6-5ca9-45b9-a4cc-66584cffed9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.769891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbc3a6-5ca9-45b9-a4cc-66584cffed9f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.770176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbc3a6-5ca9-45b9-a4cc-66584cffed9f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.770450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbc3a6-5ca9-45b9-a4cc-66584cffed9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.771201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbc3a6-5ca9-45b9-a4cc-66584cffed9f None admin] RBAC: Adding query filter params (role.id=0a24fcf372404d29a48ce7ff77aac174, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=20dd03dd2fa04147bba827f983a58034) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:07.771508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbc3a6-5ca9-45b9-a4cc-66584cffed9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.772966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbc3a6-5ca9-45b9-a4cc-66584cffed9f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.789882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 544/1106] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/role_assignments?user.id=20dd03dd2fa04147bba827f983a58034&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=0a24fcf372404d29a48ce7ff77aac174 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:09.696344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429e4b1-a31c-4b45-8861-3fd0e233351e None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:09.696877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429e4b1-a31c-4b45-8861-3fd0e233351e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:09.696877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429e4b1-a31c-4b45-8861-3fd0e233351e None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:09.698272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 563/1107] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:09.703577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634738-fafc-4698-b528-9158deee9814 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:09.703954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634738-fafc-4698-b528-9158deee9814 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:09.704245 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634738-fafc-4698-b528-9158deee9814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:09.722617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-bd634738-fafc-4698-b528-9158deee9814 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:09.723813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bd634738-fafc-4698-b528-9158deee9814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:09.789740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 545/1108] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:10.069617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429e4b1-a31c-4b45-8861-3fd0e233351e None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:10.171180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429e4b1-a31c-4b45-8861-3fd0e233351e None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:10.209921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37163c54-6837-49a6-b439-12e03cb8e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:10.210548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37163c54-6837-49a6-b439-12e03cb8e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:10.210851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37163c54-6837-49a6-b439-12e03cb8e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:10.211253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37163c54-6837-49a6-b439-12e03cb8e2a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:10.212393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37163c54-6837-49a6-b439-12e03cb8e2a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:10.213813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37163c54-6837-49a6-b439-12e03cb8e2a9 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:10.215414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 564/1109] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:10.221779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd634738-fafc-4698-b528-9158deee9814 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:10.267719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:10.321058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd634738-fafc-4698-b528-9158deee9814 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:10.354931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f3d33-f119-41e4-b6bd-d33e99f5d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:10.357747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f3d33-f119-41e4-b6bd-d33e99f5d4a8 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:10.357747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f3d33-f119-41e4-b6bd-d33e99f5d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:10.357747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f3d33-f119-41e4-b6bd-d33e99f5d4a8 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:10.357747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f3d33-f119-41e4-b6bd-d33e99f5d4a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:10.358132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f3d33-f119-41e4-b6bd-d33e99f5d4a8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:10.658721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 546/1110] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:31:10 2022] POST /identity/v3/users => generated 286 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:12.607569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311df1e9-9861-4900-a4ee-b09d7ffd9960 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:12.608093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311df1e9-9861-4900-a4ee-b09d7ffd9960 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:12.608093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311df1e9-9861-4900-a4ee-b09d7ffd9960 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:12.609434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 565/1111] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:12.615001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbe417-c148-4eb2-aab5-1f3b2d82d351 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:12.615442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbe417-c148-4eb2-aab5-1f3b2d82d351 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:12.615538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbe417-c148-4eb2-aab5-1f3b2d82d351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:12.634231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-61dbe417-c148-4eb2-aab5-1f3b2d82d351 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:12.635418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-61dbe417-c148-4eb2-aab5-1f3b2d82d351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:12.702913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 547/1112] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:12.975026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311df1e9-9861-4900-a4ee-b09d7ffd9960 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:13.037556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:13.077427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311df1e9-9861-4900-a4ee-b09d7ffd9960 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.112562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7767e-0a5e-424d-ba5d-66e576e8fc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.113175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7767e-0a5e-424d-ba5d-66e576e8fc92 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.113451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7767e-0a5e-424d-ba5d-66e576e8fc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.113720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7767e-0a5e-424d-ba5d-66e576e8fc92 None admin] PATH_INFO: `/v3/roles/d6fe1aca91e74aad9907c855501f588f` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.115115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7767e-0a5e-424d-ba5d-66e576e8fc92 None admin] RBAC: Authorizing `identity:get_role(role_id=d6fe1aca91e74aad9907c855501f588f)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.116456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7767e-0a5e-424d-ba5d-66e576e8fc92 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.117759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 566/1113] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:12 2022] GET /identity/v3/roles/d6fe1aca91e74aad9907c855501f588f => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:13.123705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbe417-c148-4eb2-aab5-1f3b2d82d351 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:13.154358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:13.221795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbe417-c148-4eb2-aab5-1f3b2d82d351 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.255451 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302e032-cf80-4352-9ecc-eb7fd860f8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.256070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302e032-cf80-4352-9ecc-eb7fd860f8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.256348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302e032-cf80-4352-9ecc-eb7fd860f8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.256619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302e032-cf80-4352-9ecc-eb7fd860f8a8 None admin] PATH_INFO: `/v3/users/62586e6b6a2c40fa8238ed2a272b18f8` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.270939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302e032-cf80-4352-9ecc-eb7fd860f8a8 None admin] RBAC: Authorizing `identity:get_user(user_id=62586e6b6a2c40fa8238ed2a272b18f8)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.272368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302e032-cf80-4352-9ecc-eb7fd860f8a8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.273989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 548/1114] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:13 2022] GET /identity/v3/users/62586e6b6a2c40fa8238ed2a272b18f8 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:13.279854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7767e-0a5e-424d-ba5d-66e576e8fc92 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:13.381481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7767e-0a5e-424d-ba5d-66e576e8fc92 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.410231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:13.415755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83799ba9-b0fa-45e1-88f6-90326b1bddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.416357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799ba9-b0fa-45e1-88f6-90326b1bddd1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.416636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799ba9-b0fa-45e1-88f6-90326b1bddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.416904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799ba9-b0fa-45e1-88f6-90326b1bddd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.483202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83799ba9-b0fa-45e1-88f6-90326b1bddd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.484552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83799ba9-b0fa-45e1-88f6-90326b1bddd1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.582187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 567/1115] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:13.588439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302e032-cf80-4352-9ecc-eb7fd860f8a8 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:13.678600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:13.688284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302e032-cf80-4352-9ecc-eb7fd860f8a8 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.722719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e87a-5f54-4be6-af04-cfec1d3415fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.723332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e87a-5f54-4be6-af04-cfec1d3415fc None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.723609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e87a-5f54-4be6-af04-cfec1d3415fc None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.723876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e87a-5f54-4be6-af04-cfec1d3415fc None admin] PATH_INFO: `/v3/projects/bd926005f58c4ff092bcc2543d0f84f9` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.724976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4e87a-5f54-4be6-af04-cfec1d3415fc None admin] RBAC: Authorizing `identity:get_project(project_id=bd926005f58c4ff092bcc2543d0f84f9)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.726425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4e87a-5f54-4be6-af04-cfec1d3415fc None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.728037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 549/1116] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:13 2022] GET /identity/v3/projects/bd926005f58c4ff092bcc2543d0f84f9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:13.734342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83799ba9-b0fa-45e1-88f6-90326b1bddd1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:13.765012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:13.833498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83799ba9-b0fa-45e1-88f6-90326b1bddd1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.867285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649f214-9c70-4c24-bd7d-6bf410806098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.867902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649f214-9c70-4c24-bd7d-6bf410806098 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.868174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649f214-9c70-4c24-bd7d-6bf410806098 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.868438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649f214-9c70-4c24-bd7d-6bf410806098 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.869156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649f214-9c70-4c24-bd7d-6bf410806098 None admin] RBAC: Adding query filter params (role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=bd926005f58c4ff092bcc2543d0f84f9, user.id=62586e6b6a2c40fa8238ed2a272b18f8) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:13.869460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649f214-9c70-4c24-bd7d-6bf410806098 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.870895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649f214-9c70-4c24-bd7d-6bf410806098 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.876171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 568/1117] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:13 2022] GET /identity/v3/role_assignments?user.id=62586e6b6a2c40fa8238ed2a272b18f8&scope.project.id=bd926005f58c4ff092bcc2543d0f84f9&role.id=d6fe1aca91e74aad9907c855501f588f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:15.772450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7af55-3ee5-40f8-816b-c31393381020 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:15.773009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7af55-3ee5-40f8-816b-c31393381020 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:15.773009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7af55-3ee5-40f8-816b-c31393381020 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:15.774399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 550/1118] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:15.779678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed670fa-5dbd-466b-b5f9-320cd1780b26 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:15.780076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed670fa-5dbd-466b-b5f9-320cd1780b26 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:15.780242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed670fa-5dbd-466b-b5f9-320cd1780b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:15.799873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-2ed670fa-5dbd-466b-b5f9-320cd1780b26 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:15.800954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2ed670fa-5dbd-466b-b5f9-320cd1780b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:15.870038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 569/1119] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:16.150110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7af55-3ee5-40f8-816b-c31393381020 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:16.253040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7af55-3ee5-40f8-816b-c31393381020 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:16.287305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6127-df1f-4258-b592-98b4c7901759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:16.287922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6127-df1f-4258-b592-98b4c7901759 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:16.288202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6127-df1f-4258-b592-98b4c7901759 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:16.288477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6127-df1f-4258-b592-98b4c7901759 None admin] PATH_INFO: `/v3/roles/d6fe1aca91e74aad9907c855501f588f` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:16.289888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f6127-df1f-4258-b592-98b4c7901759 None admin] RBAC: Authorizing `identity:get_role(role_id=d6fe1aca91e74aad9907c855501f588f)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:16.291267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f6127-df1f-4258-b592-98b4c7901759 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:16.292594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 551/1120] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:16 2022] GET /identity/v3/roles/d6fe1aca91e74aad9907c855501f588f => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:16.298858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed670fa-5dbd-466b-b5f9-320cd1780b26 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:16.324317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:16.400083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed670fa-5dbd-466b-b5f9-320cd1780b26 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:16.435002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade3691-e255-4a66-98b2-9af3df8658cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:16.435556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3691-e255-4a66-98b2-9af3df8658cd None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:16.435838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3691-e255-4a66-98b2-9af3df8658cd None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:16.436364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3691-e255-4a66-98b2-9af3df8658cd None admin] PATH_INFO: `/v3/users/62586e6b6a2c40fa8238ed2a272b18f8` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:16.437484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade3691-e255-4a66-98b2-9af3df8658cd None admin] RBAC: Authorizing `identity:get_user(user_id=62586e6b6a2c40fa8238ed2a272b18f8)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:16.438997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade3691-e255-4a66-98b2-9af3df8658cd None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:16.441119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 570/1121] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:16 2022] GET /identity/v3/users/62586e6b6a2c40fa8238ed2a272b18f8 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:16.447313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6127-df1f-4258-b592-98b4c7901759 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:16.475525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:16.547770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6127-df1f-4258-b592-98b4c7901759 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:16.586799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699aaf9c-7cb8-4b30-a025-cf9e4537f831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:16.587425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aaf9c-7cb8-4b30-a025-cf9e4537f831 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:16.587741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aaf9c-7cb8-4b30-a025-cf9e4537f831 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:16.588012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aaf9c-7cb8-4b30-a025-cf9e4537f831 None admin] PATH_INFO: `/v3/projects/bd926005f58c4ff092bcc2543d0f84f9` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:16.589122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699aaf9c-7cb8-4b30-a025-cf9e4537f831 None admin] RBAC: Authorizing `identity:get_project(project_id=bd926005f58c4ff092bcc2543d0f84f9)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:16.590576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699aaf9c-7cb8-4b30-a025-cf9e4537f831 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:16.592210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 552/1122] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:16 2022] GET /identity/v3/projects/bd926005f58c4ff092bcc2543d0f84f9 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:16.598747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade3691-e255-4a66-98b2-9af3df8658cd None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:16.689526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:16.698620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade3691-e255-4a66-98b2-9af3df8658cd None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:16.732249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c749-8d24-4fa4-8a5e-529ef6fd919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:16.732906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c749-8d24-4fa4-8a5e-529ef6fd919f None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:16.733182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c749-8d24-4fa4-8a5e-529ef6fd919f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:16.733450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c749-8d24-4fa4-8a5e-529ef6fd919f None admin] PATH_INFO: `/v3/projects/bd926005f58c4ff092bcc2543d0f84f9/users/62586e6b6a2c40fa8238ed2a272b18f8/roles/d6fe1aca91e74aad9907c855501f588f` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:16.735582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79c749-8d24-4fa4-8a5e-529ef6fd919f None admin] RBAC: Authorizing `identity:create_grant(project_id=bd926005f58c4ff092bcc2543d0f84f9, user_id=62586e6b6a2c40fa8238ed2a272b18f8, role_id=d6fe1aca91e74aad9907c855501f588f)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:16.737919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79c749-8d24-4fa4-8a5e-529ef6fd919f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:16.811696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 571/1123] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:31:16 2022] PUT /identity/v3/projects/bd926005f58c4ff092bcc2543d0f84f9/users/62586e6b6a2c40fa8238ed2a272b18f8/roles/d6fe1aca91e74aad9907c855501f588f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:18.753488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf437-07a1-4069-ad68-e6e7169acb06 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:18.754061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf437-07a1-4069-ad68-e6e7169acb06 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:18.754061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf437-07a1-4069-ad68-e6e7169acb06 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:18.755415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 553/1124] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:18.760825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3b99-c4c3-415b-a005-964d3e0c998a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:18.761228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3b99-c4c3-415b-a005-964d3e0c998a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:18.761374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3b99-c4c3-415b-a005-964d3e0c998a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:18.779447 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3daf3b99-c4c3-415b-a005-964d3e0c998a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:18.780643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3daf3b99-c4c3-415b-a005-964d3e0c998a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:18.849347 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 572/1125] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:19.118994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf437-07a1-4069-ad68-e6e7169acb06 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:19.214475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:19.222786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf437-07a1-4069-ad68-e6e7169acb06 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:19.257523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7271b7-283f-42f3-bb81-8619f6f7260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:19.258181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7271b7-283f-42f3-bb81-8619f6f7260b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:19.258464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7271b7-283f-42f3-bb81-8619f6f7260b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:19.258739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7271b7-283f-42f3-bb81-8619f6f7260b None admin] PATH_INFO: `/v3/roles/d6fe1aca91e74aad9907c855501f588f` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:19.260162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7271b7-283f-42f3-bb81-8619f6f7260b None admin] RBAC: Authorizing `identity:get_role(role_id=d6fe1aca91e74aad9907c855501f588f)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:19.261531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7271b7-283f-42f3-bb81-8619f6f7260b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:19.262896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 554/1126] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:19 2022] GET /identity/v3/roles/d6fe1aca91e74aad9907c855501f588f => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:19.269537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf3b99-c4c3-415b-a005-964d3e0c998a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:19.350239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:19.367605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf3b99-c4c3-415b-a005-964d3e0c998a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:19.401977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b44a99-2119-4d33-918f-00f325e7f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:19.402535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44a99-2119-4d33-918f-00f325e7f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:19.402814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44a99-2119-4d33-918f-00f325e7f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:19.403087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44a99-2119-4d33-918f-00f325e7f2c1 None admin] PATH_INFO: `/v3/users/62586e6b6a2c40fa8238ed2a272b18f8` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:19.404198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b44a99-2119-4d33-918f-00f325e7f2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=62586e6b6a2c40fa8238ed2a272b18f8)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:19.405693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b44a99-2119-4d33-918f-00f325e7f2c1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:19.407794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 573/1127] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:19 2022] GET /identity/v3/users/62586e6b6a2c40fa8238ed2a272b18f8 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:19.413578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7271b7-283f-42f3-bb81-8619f6f7260b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:19.513087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7271b7-283f-42f3-bb81-8619f6f7260b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:19.546817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a1f8c-066b-4b9d-a5e9-87e910d08ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:19.547409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1f8c-066b-4b9d-a5e9-87e910d08ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:19.547686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1f8c-066b-4b9d-a5e9-87e910d08ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:19.548023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1f8c-066b-4b9d-a5e9-87e910d08ab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:19.578668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:19.618341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a1f8c-066b-4b9d-a5e9-87e910d08ab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:19.619784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a1f8c-066b-4b9d-a5e9-87e910d08ab7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:19.718813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 555/1128] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:19.725187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b44a99-2119-4d33-918f-00f325e7f2c1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:19.826402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b44a99-2119-4d33-918f-00f325e7f2c1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:19.859410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f0d8-a6fc-4dfa-aa41-32c51614a4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:19.860032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f0d8-a6fc-4dfa-aa41-32c51614a4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:19.860314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f0d8-a6fc-4dfa-aa41-32c51614a4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:19.860588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f0d8-a6fc-4dfa-aa41-32c51614a4a5 None admin] PATH_INFO: `/v3/projects/bd926005f58c4ff092bcc2543d0f84f9` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:19.862338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f0d8-a6fc-4dfa-aa41-32c51614a4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=bd926005f58c4ff092bcc2543d0f84f9)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:19.863769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f0d8-a6fc-4dfa-aa41-32c51614a4a5 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:19.865414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 574/1129] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:19 2022] GET /identity/v3/projects/bd926005f58c4ff092bcc2543d0f84f9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:19.871813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a1f8c-066b-4b9d-a5e9-87e910d08ab7 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:19.934047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:19.973574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a1f8c-066b-4b9d-a5e9-87e910d08ab7 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:20.007315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42132db-4157-4b3a-a31a-4672ef67c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:20.007912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42132db-4157-4b3a-a31a-4672ef67c042 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:20.008193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42132db-4157-4b3a-a31a-4672ef67c042 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:20.008470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42132db-4157-4b3a-a31a-4672ef67c042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:20.009197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42132db-4157-4b3a-a31a-4672ef67c042 None admin] RBAC: Adding query filter params (role.id=d6fe1aca91e74aad9907c855501f588f, scope.project.id=bd926005f58c4ff092bcc2543d0f84f9, user.id=62586e6b6a2c40fa8238ed2a272b18f8) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:20.009507 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42132db-4157-4b3a-a31a-4672ef67c042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:20.011000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42132db-4157-4b3a-a31a-4672ef67c042 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:20.016973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 556/1130] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:19 2022] GET /identity/v3/role_assignments?user.id=62586e6b6a2c40fa8238ed2a272b18f8&scope.project.id=bd926005f58c4ff092bcc2543d0f84f9&role.id=d6fe1aca91e74aad9907c855501f588f => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:21.977870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93483-1514-4b74-b2f1-a46039333ad0 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:21.978371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93483-1514-4b74-b2f1-a46039333ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:21.978486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93483-1514-4b74-b2f1-a46039333ad0 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:21.979761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 575/1131] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:21.985119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d3f4c-f113-4356-9156-f170ddbc3576 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:21.985527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d3f4c-f113-4356-9156-f170ddbc3576 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:21.985685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d3f4c-f113-4356-9156-f170ddbc3576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:22.003641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-cc4d3f4c-f113-4356-9156-f170ddbc3576 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:22.004828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cc4d3f4c-f113-4356-9156-f170ddbc3576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:22.072450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 557/1132] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:22.345440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f93483-1514-4b74-b2f1-a46039333ad0 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:22.365846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:22.450098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f93483-1514-4b74-b2f1-a46039333ad0 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:22.484884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972b566-743a-4dd4-9ea6-278461d107a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:22.485507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b566-743a-4dd4-9ea6-278461d107a2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:22.485785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b566-743a-4dd4-9ea6-278461d107a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:22.486086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b566-743a-4dd4-9ea6-278461d107a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:22.487895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972b566-743a-4dd4-9ea6-278461d107a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:22.489314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972b566-743a-4dd4-9ea6-278461d107a2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:22.490937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 576/1133] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:22.499535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d3f4c-f113-4356-9156-f170ddbc3576 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:22.556946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:22.603067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d3f4c-f113-4356-9156-f170ddbc3576 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:22.636507 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf4ad4-057f-4529-ac89-5158c0bd067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:22.637106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf4ad4-057f-4529-ac89-5158c0bd067a None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:22.637484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf4ad4-057f-4529-ac89-5158c0bd067a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:22.637704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf4ad4-057f-4529-ac89-5158c0bd067a None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:22.638319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf4ad4-057f-4529-ac89-5158c0bd067a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:22.639782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf4ad4-057f-4529-ac89-5158c0bd067a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:22.732730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 558/1134] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:31:22 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:24.745887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0388301-0706-4f65-bbcc-844d196fddc1 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:24.746419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0388301-0706-4f65-bbcc-844d196fddc1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:24.746419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0388301-0706-4f65-bbcc-844d196fddc1 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:24.747805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 577/1135] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:24.753081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054813f-3a86-49e6-ba18-94c8a4f49d35 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:24.753480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054813f-3a86-49e6-ba18-94c8a4f49d35 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:24.753635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054813f-3a86-49e6-ba18-94c8a4f49d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:24.771527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d054813f-3a86-49e6-ba18-94c8a4f49d35 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:24.772716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d054813f-3a86-49e6-ba18-94c8a4f49d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:24.840232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 559/1136] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:25.110150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0388301-0706-4f65-bbcc-844d196fddc1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:25.197497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:25.212778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0388301-0706-4f65-bbcc-844d196fddc1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:25.247191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfb3d1-3583-4d70-b248-e8eb1289ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:25.247811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfb3d1-3583-4d70-b248-e8eb1289ae71 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:25.248089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfb3d1-3583-4d70-b248-e8eb1289ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:25.248358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfb3d1-3583-4d70-b248-e8eb1289ae71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:25.249472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfb3d1-3583-4d70-b248-e8eb1289ae71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:25.250925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfb3d1-3583-4d70-b248-e8eb1289ae71 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:25.252560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 578/1137] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:25.258461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054813f-3a86-49e6-ba18-94c8a4f49d35 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:25.297894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:25.358360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054813f-3a86-49e6-ba18-94c8a4f49d35 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:25.393296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc34e82-772a-4a2e-acf3-9a194f601f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:25.393886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc34e82-772a-4a2e-acf3-9a194f601f63 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:25.394170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc34e82-772a-4a2e-acf3-9a194f601f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:25.394443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc34e82-772a-4a2e-acf3-9a194f601f63 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:25.395083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc34e82-772a-4a2e-acf3-9a194f601f63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:25.396516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc34e82-772a-4a2e-acf3-9a194f601f63 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:25.520383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 560/1138] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:31:25 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:27.481860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62041-4fc3-4bc0-9c06-26278f44464b None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:27.482393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62041-4fc3-4bc0-9c06-26278f44464b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:27.482527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62041-4fc3-4bc0-9c06-26278f44464b None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:27.483781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 579/1139] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:27.489839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5942487-050c-4322-8f5f-0fb97748c34f None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:27.490281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5942487-050c-4322-8f5f-0fb97748c34f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:27.490426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5942487-050c-4322-8f5f-0fb97748c34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:27.509150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e5942487-050c-4322-8f5f-0fb97748c34f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:27.510394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e5942487-050c-4322-8f5f-0fb97748c34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:27.576705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 561/1140] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:27.851275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd62041-4fc3-4bc0-9c06-26278f44464b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:27.952175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd62041-4fc3-4bc0-9c06-26278f44464b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:27.986283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708e9bd-7b7f-4139-92ca-cb24aac8f97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:27.986915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708e9bd-7b7f-4139-92ca-cb24aac8f97e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:27.987205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708e9bd-7b7f-4139-92ca-cb24aac8f97e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:27.987480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708e9bd-7b7f-4139-92ca-cb24aac8f97e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:27.997517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708e9bd-7b7f-4139-92ca-cb24aac8f97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-b708e9bd-7b7f-4139-92ca-cb24aac8f97e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:28.002223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:28.007708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:28.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 580/1141] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:28.009395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5942487-050c-4322-8f5f-0fb97748c34f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:28.035461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:28.109570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5942487-050c-4322-8f5f-0fb97748c34f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:28.144199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:28.144794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:28.145070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:28.150363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:28.150363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:28.150363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:28.150363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:28.155549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 562/1142] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:28.162223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708e9bd-7b7f-4139-92ca-cb24aac8f97e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:28.178596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:28.263364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708e9bd-7b7f-4139-92ca-cb24aac8f97e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:28.297649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732cfc6-bc95-41a9-91ae-f3e187d08a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:28.298293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732cfc6-bc95-41a9-91ae-f3e187d08a0d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:28.298608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732cfc6-bc95-41a9-91ae-f3e187d08a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:28.298887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732cfc6-bc95-41a9-91ae-f3e187d08a0d None admin] PATH_INFO: `/v3/users/3314c6ae4a0349b3b4b1623b0875df35` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:28.312650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732cfc6-bc95-41a9-91ae-f3e187d08a0d None admin] RBAC: Authorizing `identity:get_user(user_id=3314c6ae4a0349b3b4b1623b0875df35)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:28.314654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732cfc6-bc95-41a9-91ae-f3e187d08a0d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:28.316301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 581/1143] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:28 2022] GET /identity/v3/users/3314c6ae4a0349b3b4b1623b0875df35 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:28.322345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:28.407308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:28.422172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc27eb-537f-4014-bc35-dd30bee47080 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:28.457904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583edac-2f3e-40c2-b845-21651655c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:28.458512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583edac-2f3e-40c2-b845-21651655c67e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:28.458795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583edac-2f3e-40c2-b845-21651655c67e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:28.459071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583edac-2f3e-40c2-b845-21651655c67e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:28.526098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583edac-2f3e-40c2-b845-21651655c67e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:28.527540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583edac-2f3e-40c2-b845-21651655c67e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:28.610316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:28.625814 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 563/1144] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:28.632376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732cfc6-bc95-41a9-91ae-f3e187d08a0d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:28.703000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:28.732317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732cfc6-bc95-41a9-91ae-f3e187d08a0d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:28.768006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c9bf8-44ac-4e56-8445-d86be512cebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:28.768633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9bf8-44ac-4e56-8445-d86be512cebe None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:28.768919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9bf8-44ac-4e56-8445-d86be512cebe None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:28.769198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9bf8-44ac-4e56-8445-d86be512cebe None admin] PATH_INFO: `/v3/projects/c0da4ba06ee44238804af05dd41ea4e0` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:28.770342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c9bf8-44ac-4e56-8445-d86be512cebe None admin] RBAC: Authorizing `identity:get_project(project_id=c0da4ba06ee44238804af05dd41ea4e0)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:28.771770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c9bf8-44ac-4e56-8445-d86be512cebe None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:28.773389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 582/1145] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:28 2022] GET /identity/v3/projects/c0da4ba06ee44238804af05dd41ea4e0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:28.779712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583edac-2f3e-40c2-b845-21651655c67e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:28.878653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583edac-2f3e-40c2-b845-21651655c67e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:28.911827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989d7bc-3fa1-40cd-b9b7-6ebd4025133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:28.912425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d7bc-3fa1-40cd-b9b7-6ebd4025133f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:28.912705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d7bc-3fa1-40cd-b9b7-6ebd4025133f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:28.912978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d7bc-3fa1-40cd-b9b7-6ebd4025133f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:28.913711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989d7bc-3fa1-40cd-b9b7-6ebd4025133f None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=c0da4ba06ee44238804af05dd41ea4e0, user.id=3314c6ae4a0349b3b4b1623b0875df35) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:28.914051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989d7bc-3fa1-40cd-b9b7-6ebd4025133f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:28.915480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989d7bc-3fa1-40cd-b9b7-6ebd4025133f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:28.921056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 564/1146] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:28 2022] GET /identity/v3/role_assignments?user.id=3314c6ae4a0349b3b4b1623b0875df35&scope.project.id=c0da4ba06ee44238804af05dd41ea4e0&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:30.819276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959b1b-2cd4-4b83-9f37-c7752a31f937 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:30.819808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959b1b-2cd4-4b83-9f37-c7752a31f937 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:30.819808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959b1b-2cd4-4b83-9f37-c7752a31f937 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:30.821156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 583/1147] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:30.826494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7374c-54c2-44c1-b616-8f07971c1974 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:30.826893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7374c-54c2-44c1-b616-8f07971c1974 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:30.827157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7374c-54c2-44c1-b616-8f07971c1974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:30.845943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-bee7374c-54c2-44c1-b616-8f07971c1974 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:30.847167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bee7374c-54c2-44c1-b616-8f07971c1974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:30.910958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:30.931654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 565/1148] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:31.202415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d959b1b-2cd4-4b83-9f37-c7752a31f937 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:31.302752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d959b1b-2cd4-4b83-9f37-c7752a31f937 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:31.337087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32bdfa-c59a-4fdb-aeee-bc60b582524c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:31.337708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32bdfa-c59a-4fdb-aeee-bc60b582524c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:31.338017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32bdfa-c59a-4fdb-aeee-bc60b582524c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:31.338291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32bdfa-c59a-4fdb-aeee-bc60b582524c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:31.345880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:31.348813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32bdfa-c59a-4fdb-aeee-bc60b582524c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-6d32bdfa-c59a-4fdb-aeee-bc60b582524c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:31.353526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:31.363778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:31.365020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 584/1149] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:31.365997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7374c-54c2-44c1-b616-8f07971c1974 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:31.460816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7374c-54c2-44c1-b616-8f07971c1974 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:31.494892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:31.495493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:31.495830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:31.496104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:31.496774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:31.497080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:31.498475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:31.507101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 566/1150] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:31.513067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32bdfa-c59a-4fdb-aeee-bc60b582524c None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:31.614588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32bdfa-c59a-4fdb-aeee-bc60b582524c None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:31.648501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8236-abee-4804-b635-45ee5eb00d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:31.649128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8236-abee-4804-b635-45ee5eb00d12 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:31.649408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8236-abee-4804-b635-45ee5eb00d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:31.649701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8236-abee-4804-b635-45ee5eb00d12 None admin] PATH_INFO: `/v3/users/3314c6ae4a0349b3b4b1623b0875df35` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:31.650858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f8236-abee-4804-b635-45ee5eb00d12 None admin] RBAC: Authorizing `identity:get_user(user_id=3314c6ae4a0349b3b4b1623b0875df35)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:31.652281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f8236-abee-4804-b635-45ee5eb00d12 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:31.653871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 585/1151] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:31 2022] GET /identity/v3/users/3314c6ae4a0349b3b4b1623b0875df35 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:31.659911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:31.704228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:31.761586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc477ef-9643-46be-ba8d-b05dd4a3ab9b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:31.794353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7dc0a-4399-4f44-82fe-06191afcc1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:31.794967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7dc0a-4399-4f44-82fe-06191afcc1db None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:31.795241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7dc0a-4399-4f44-82fe-06191afcc1db None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:31.795509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7dc0a-4399-4f44-82fe-06191afcc1db None admin] PATH_INFO: `/v3/projects/c0da4ba06ee44238804af05dd41ea4e0` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:31.796609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7dc0a-4399-4f44-82fe-06191afcc1db None admin] RBAC: Authorizing `identity:get_project(project_id=c0da4ba06ee44238804af05dd41ea4e0)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:31.798059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7dc0a-4399-4f44-82fe-06191afcc1db None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:31.800517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 567/1152] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:31 2022] GET /identity/v3/projects/c0da4ba06ee44238804af05dd41ea4e0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:31.806695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8236-abee-4804-b635-45ee5eb00d12 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:31.879767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:31.911599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8236-abee-4804-b635-45ee5eb00d12 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:31.947051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092682a-e988-404c-9aaf-d15f019fd10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:31.947720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092682a-e988-404c-9aaf-d15f019fd10f None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:31.947999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092682a-e988-404c-9aaf-d15f019fd10f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:31.948270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092682a-e988-404c-9aaf-d15f019fd10f None admin] PATH_INFO: `/v3/projects/c0da4ba06ee44238804af05dd41ea4e0/users/3314c6ae4a0349b3b4b1623b0875df35/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:31.950181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092682a-e988-404c-9aaf-d15f019fd10f None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da4ba06ee44238804af05dd41ea4e0, user_id=3314c6ae4a0349b3b4b1623b0875df35, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:31.951637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092682a-e988-404c-9aaf-d15f019fd10f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:32.035191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 586/1153] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:31:31 2022] PUT /identity/v3/projects/c0da4ba06ee44238804af05dd41ea4e0/users/3314c6ae4a0349b3b4b1623b0875df35/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:33.976745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ede50a-28a6-4b6b-8e90-8179e427f98a None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:33.977331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ede50a-28a6-4b6b-8e90-8179e427f98a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:33.977331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ede50a-28a6-4b6b-8e90-8179e427f98a None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:33.978656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 568/1154] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:33.983940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad278d-5b50-482f-b8b5-a21ae4db97a1 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:33.984357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad278d-5b50-482f-b8b5-a21ae4db97a1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:33.984509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad278d-5b50-482f-b8b5-a21ae4db97a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:34.002922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-2bad278d-5b50-482f-b8b5-a21ae4db97a1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:34.009146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2bad278d-5b50-482f-b8b5-a21ae4db97a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:34.071702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 587/1155] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:34.349802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ede50a-28a6-4b6b-8e90-8179e427f98a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:34.451069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ede50a-28a6-4b6b-8e90-8179e427f98a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:34.461210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:34.484328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2375-6878-459c-9ee4-e85f9f8eed0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:34.484936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2375-6878-459c-9ee4-e85f9f8eed0c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:34.485212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2375-6878-459c-9ee4-e85f9f8eed0c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:34.485478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2375-6878-459c-9ee4-e85f9f8eed0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:34.496911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec2375-6878-459c-9ee4-e85f9f8eed0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-10ec2375-6878-459c-9ee4-e85f9f8eed0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:34.501931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:34.503160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:34.504268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 569/1156] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:34.509084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad278d-5b50-482f-b8b5-a21ae4db97a1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:34.568395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:34.610117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad278d-5b50-482f-b8b5-a21ae4db97a1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:34.643645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:34.644277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:34.644557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:34.644829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:34.645494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:34.645798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:34.647186 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:34.655051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 588/1157] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:34.661609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2375-6878-459c-9ee4-e85f9f8eed0c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:34.760429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2375-6878-459c-9ee4-e85f9f8eed0c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:34.793803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2d95b-3b96-4e21-b124-ec6805352d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:34.794461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d95b-3b96-4e21-b124-ec6805352d48 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:34.794740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d95b-3b96-4e21-b124-ec6805352d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:34.795013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d95b-3b96-4e21-b124-ec6805352d48 None admin] PATH_INFO: `/v3/users/3314c6ae4a0349b3b4b1623b0875df35` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:34.796146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2d95b-3b96-4e21-b124-ec6805352d48 None admin] RBAC: Authorizing `identity:get_user(user_id=3314c6ae4a0349b3b4b1623b0875df35)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:34.797573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2d95b-3b96-4e21-b124-ec6805352d48 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:34.799359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 570/1158] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:34 2022] GET /identity/v3/users/3314c6ae4a0349b3b4b1623b0875df35 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:34.805335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:34.904848 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae50c7-66e2-4c51-b88f-8822b817d836 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:34.927460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:34.941340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf66d-906f-4eec-9278-23256cdb59b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:34.941977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf66d-906f-4eec-9278-23256cdb59b8 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:34.942255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf66d-906f-4eec-9278-23256cdb59b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:34.942524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf66d-906f-4eec-9278-23256cdb59b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:35.009194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cf66d-906f-4eec-9278-23256cdb59b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:35.010706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cf66d-906f-4eec-9278-23256cdb59b8 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:35.108239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 589/1159] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:31:35.114345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2d95b-3b96-4e21-b124-ec6805352d48 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:35.154596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:35.215312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2d95b-3b96-4e21-b124-ec6805352d48 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:35.248155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3bd71-060f-4781-863f-a65ab8c47bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:35.248772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3bd71-060f-4781-863f-a65ab8c47bcb None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:35.249052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3bd71-060f-4781-863f-a65ab8c47bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:35.249328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3bd71-060f-4781-863f-a65ab8c47bcb None admin] PATH_INFO: `/v3/projects/c0da4ba06ee44238804af05dd41ea4e0` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:35.250592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3bd71-060f-4781-863f-a65ab8c47bcb None admin] RBAC: Authorizing `identity:get_project(project_id=c0da4ba06ee44238804af05dd41ea4e0)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:35.252749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3bd71-060f-4781-863f-a65ab8c47bcb None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:35.254400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 571/1160] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:35 2022] GET /identity/v3/projects/c0da4ba06ee44238804af05dd41ea4e0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:35.260586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf66d-906f-4eec-9278-23256cdb59b8 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:35.282073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:35.359503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf66d-906f-4eec-9278-23256cdb59b8 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:35.393122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2d0bb-fa3d-4777-b4fd-dd4f2f23b8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:35.393706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2d0bb-fa3d-4777-b4fd-dd4f2f23b8cc None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:35.394014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2d0bb-fa3d-4777-b4fd-dd4f2f23b8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:35.394284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2d0bb-fa3d-4777-b4fd-dd4f2f23b8cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:35.394998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2d0bb-fa3d-4777-b4fd-dd4f2f23b8cc None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=c0da4ba06ee44238804af05dd41ea4e0, user.id=3314c6ae4a0349b3b4b1623b0875df35) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:35.395301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2d0bb-fa3d-4777-b4fd-dd4f2f23b8cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:35.396710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2d0bb-fa3d-4777-b4fd-dd4f2f23b8cc None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:35.402318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 590/1161] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:35 2022] GET /identity/v3/role_assignments?user.id=3314c6ae4a0349b3b4b1623b0875df35&scope.project.id=c0da4ba06ee44238804af05dd41ea4e0&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:37.348740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3631e-ff5d-4689-841a-bd5c03372f94 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:37.349274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3631e-ff5d-4689-841a-bd5c03372f94 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:37.349274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3631e-ff5d-4689-841a-bd5c03372f94 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:37.350692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 572/1162] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:37.356273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1eaa-6ca5-467e-bd26-23a2a96d9e6a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:37.356709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1eaa-6ca5-467e-bd26-23a2a96d9e6a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:37.356830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1eaa-6ca5-467e-bd26-23a2a96d9e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:37.375587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-2b9a1eaa-6ca5-467e-bd26-23a2a96d9e6a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:37.376840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2b9a1eaa-6ca5-467e-bd26-23a2a96d9e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:37.442815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 591/1163] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:37.717951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3631e-ff5d-4689-841a-bd5c03372f94 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:37.820731 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3631e-ff5d-4689-841a-bd5c03372f94 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:37.827668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:37.855759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7eadf-3147-433c-9e00-08f0cc31e5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:37.856382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7eadf-3147-433c-9e00-08f0cc31e5df None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:37.856659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7eadf-3147-433c-9e00-08f0cc31e5df None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:37.856927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7eadf-3147-433c-9e00-08f0cc31e5df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:37.927162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7eadf-3147-433c-9e00-08f0cc31e5df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:37.928655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7eadf-3147-433c-9e00-08f0cc31e5df None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:38.028692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 573/1164] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:38.034797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1eaa-6ca5-467e-bd26-23a2a96d9e6a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:38.049820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:38.133346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1eaa-6ca5-467e-bd26-23a2a96d9e6a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:38.167177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797c911-c2a8-40c2-9a4f-619bcc758584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:38.167796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797c911-c2a8-40c2-9a4f-619bcc758584 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:38.168072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797c911-c2a8-40c2-9a4f-619bcc758584 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:38.168339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797c911-c2a8-40c2-9a4f-619bcc758584 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:38.175109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797c911-c2a8-40c2-9a4f-619bcc758584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:38.176526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797c911-c2a8-40c2-9a4f-619bcc758584 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-8797c911-c2a8-40c2-9a4f-619bcc758584 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:38.187595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:38.189306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:38.190463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 592/1165] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:31:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:38.195481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7eadf-3147-433c-9e00-08f0cc31e5df None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:38.203895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:38.295536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7eadf-3147-433c-9e00-08f0cc31e5df None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:38.329861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41447ff5-1b88-4822-b5b4-24851d088a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:38.330497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447ff5-1b88-4822-b5b4-24851d088a8c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:38.330786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447ff5-1b88-4822-b5b4-24851d088a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:38.331062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447ff5-1b88-4822-b5b4-24851d088a8c None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:38.331705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41447ff5-1b88-4822-b5b4-24851d088a8c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:38.332012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41447ff5-1b88-4822-b5b4-24851d088a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:38.333374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41447ff5-1b88-4822-b5b4-24851d088a8c None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:38.340457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 574/1166] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 17:31:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:40.245553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d4a3-3e02-404d-aa6b-fbb462b3e93a None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:40.246124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d4a3-3e02-404d-aa6b-fbb462b3e93a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:40.246247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d4a3-3e02-404d-aa6b-fbb462b3e93a None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:40.247601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 593/1167] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:40.253585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902d5b-4c20-4224-917c-09d34af8c0ac None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:40.254029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902d5b-4c20-4224-917c-09d34af8c0ac None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:40.254174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902d5b-4c20-4224-917c-09d34af8c0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:40.273247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e3902d5b-4c20-4224-917c-09d34af8c0ac None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:40.274498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e3902d5b-4c20-4224-917c-09d34af8c0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:40.336147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:40.340708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 575/1168] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:40.617228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52d4a3-3e02-404d-aa6b-fbb462b3e93a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:40.693599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:40.718499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52d4a3-3e02-404d-aa6b-fbb462b3e93a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:40.753902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495221-1f75-4bd1-a2d8-d789c0df0a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:40.754501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495221-1f75-4bd1-a2d8-d789c0df0a7e None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:40.754784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495221-1f75-4bd1-a2d8-d789c0df0a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:40.755060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495221-1f75-4bd1-a2d8-d789c0df0a7e None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:40.755729 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495221-1f75-4bd1-a2d8-d789c0df0a7e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:40.757084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495221-1f75-4bd1-a2d8-d789c0df0a7e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:40.852168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 594/1169] 10.222.0.76 () {60 vars in 1138 bytes} [Wed Aug 24 17:31:40 2022] POST /identity/v3/domains => generated 267 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:42.805566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433e892-fa09-410a-a729-4cb328c8aebe None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:42.806177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433e892-fa09-410a-a729-4cb328c8aebe None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:42.806177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433e892-fa09-410a-a729-4cb328c8aebe None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:42.807495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 576/1170] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:42.813154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39822e88-04da-4696-94a6-57bdd16f2216 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:42.813666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39822e88-04da-4696-94a6-57bdd16f2216 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:42.813974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39822e88-04da-4696-94a6-57bdd16f2216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:42.833608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-39822e88-04da-4696-94a6-57bdd16f2216 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:42.834834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-39822e88-04da-4696-94a6-57bdd16f2216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:42.900914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 595/1171] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:43.173198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433e892-fa09-410a-a729-4cb328c8aebe None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:43.274443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433e892-fa09-410a-a729-4cb328c8aebe None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:43.310379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb078c-fc6e-4bbf-80cb-e9d96fc7e567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:43.311002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb078c-fc6e-4bbf-80cb-e9d96fc7e567 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:43.311285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb078c-fc6e-4bbf-80cb-e9d96fc7e567 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:43.311557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb078c-fc6e-4bbf-80cb-e9d96fc7e567 None admin] PATH_INFO: `/v3/domains/cff1c8c73e3d40d4a677a415bd4cf3ba` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:43.322752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb078c-fc6e-4bbf-80cb-e9d96fc7e567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cff1c8c73e3d40d4a677a415bd4cf3ba)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:43.324276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb078c-fc6e-4bbf-80cb-e9d96fc7e567 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:43.325922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 577/1172] 10.222.0.76 () {56 vars in 1183 bytes} [Wed Aug 24 17:31:43 2022] GET /identity/v3/domains/cff1c8c73e3d40d4a677a415bd4cf3ba => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:43.332215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39822e88-04da-4696-94a6-57bdd16f2216 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:43.393936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:43.432334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39822e88-04da-4696-94a6-57bdd16f2216 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:43.465874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca688c82-58ba-408d-b7ef-66e0844b7651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:43.466476 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688c82-58ba-408d-b7ef-66e0844b7651 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:43.466757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688c82-58ba-408d-b7ef-66e0844b7651 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:43.467030 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688c82-58ba-408d-b7ef-66e0844b7651 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:43.467682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca688c82-58ba-408d-b7ef-66e0844b7651 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:43.469141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca688c82-58ba-408d-b7ef-66e0844b7651 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:43.589652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 596/1173] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:31:43 2022] POST /identity/v3/projects => generated 361 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:31:45.518801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f50d19-dbb0-4577-834a-a1284e5711a5 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:45.519364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f50d19-dbb0-4577-834a-a1284e5711a5 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:45.519364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f50d19-dbb0-4577-834a-a1284e5711a5 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:45.520717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 578/1174] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:45.526007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582d12-0b21-4433-9092-6b27dbeccab2 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:45.526436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582d12-0b21-4433-9092-6b27dbeccab2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:45.526565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582d12-0b21-4433-9092-6b27dbeccab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:45.544309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-69582d12-0b21-4433-9092-6b27dbeccab2 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:45.545494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-69582d12-0b21-4433-9092-6b27dbeccab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:45.612194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 597/1175] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:45.883514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f50d19-dbb0-4577-834a-a1284e5711a5 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:45.945926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:45.986605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f50d19-dbb0-4577-834a-a1284e5711a5 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:46.020116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55810dcc-df37-46e2-ab43-c6f3ebbe373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:46.020731 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810dcc-df37-46e2-ab43-c6f3ebbe373e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:46.021015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810dcc-df37-46e2-ab43-c6f3ebbe373e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:46.021316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810dcc-df37-46e2-ab43-c6f3ebbe373e None admin] PATH_INFO: `/v3/domains/cff1c8c73e3d40d4a677a415bd4cf3ba` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:46.022435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55810dcc-df37-46e2-ab43-c6f3ebbe373e None admin] RBAC: Authorizing `identity:get_domain(domain_id=cff1c8c73e3d40d4a677a415bd4cf3ba)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:46.023858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55810dcc-df37-46e2-ab43-c6f3ebbe373e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:46.025411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 579/1176] 10.222.0.76 () {56 vars in 1183 bytes} [Wed Aug 24 17:31:45 2022] GET /identity/v3/domains/cff1c8c73e3d40d4a677a415bd4cf3ba => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:46.031214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69582d12-0b21-4433-9092-6b27dbeccab2 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:46.078135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:46.131162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69582d12-0b21-4433-9092-6b27dbeccab2 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:46.165289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebd2ed-0d48-4254-968e-1486dec5b527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:46.165820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd2ed-0d48-4254-968e-1486dec5b527 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:46.166128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd2ed-0d48-4254-968e-1486dec5b527 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:46.166397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd2ed-0d48-4254-968e-1486dec5b527 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:46.167024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebd2ed-0d48-4254-968e-1486dec5b527 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:46.168435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebd2ed-0d48-4254-968e-1486dec5b527 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:46.268925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 598/1177] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:31:46 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:48.202392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20e040-cd54-4b4b-9191-165cb003c4a9 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:48.202891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20e040-cd54-4b4b-9191-165cb003c4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:48.202891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20e040-cd54-4b4b-9191-165cb003c4a9 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:48.204601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 580/1178] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:48.210048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9771-6ba7-4491-9db3-0604bf1e6a13 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:48.210428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9771-6ba7-4491-9db3-0604bf1e6a13 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:48.210603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9771-6ba7-4491-9db3-0604bf1e6a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:48.228413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-5d8d9771-6ba7-4491-9db3-0604bf1e6a13 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:48.229593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5d8d9771-6ba7-4491-9db3-0604bf1e6a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:48.296181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 599/1179] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:48.560795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20e040-cd54-4b4b-9191-165cb003c4a9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:48.663740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20e040-cd54-4b4b-9191-165cb003c4a9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:48.689992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:48.697502 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a60a85-ba4b-4893-8c18-3a9dcbff555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:48.698159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60a85-ba4b-4893-8c18-3a9dcbff555f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:48.698443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60a85-ba4b-4893-8c18-3a9dcbff555f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:48.698717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60a85-ba4b-4893-8c18-3a9dcbff555f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:48.710098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a60a85-ba4b-4893-8c18-3a9dcbff555f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-11a60a85-ba4b-4893-8c18-3a9dcbff555f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:48.714832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:48.715939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:48.717131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 581/1180] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:48.722130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d9771-6ba7-4491-9db3-0604bf1e6a13 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:48.752469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:48.819638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d9771-6ba7-4491-9db3-0604bf1e6a13 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:48.852935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:48.853526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:48.853807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:48.854114 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:48.854786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:48.855092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:48.856441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:48.864821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 600/1181] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:48.870718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a60a85-ba4b-4893-8c18-3a9dcbff555f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:48.970401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a60a85-ba4b-4893-8c18-3a9dcbff555f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:49.004027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279609e1-cc63-4847-b499-078ce7ddc7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:49.009413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279609e1-cc63-4847-b499-078ce7ddc7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:49.009413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279609e1-cc63-4847-b499-078ce7ddc7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:49.009413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279609e1-cc63-4847-b499-078ce7ddc7e5 None admin] PATH_INFO: `/v3/users/e5b59dffce894dbbaafbe1eeff42aabb` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:49.018492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279609e1-cc63-4847-b499-078ce7ddc7e5 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b59dffce894dbbaafbe1eeff42aabb)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:49.019926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279609e1-cc63-4847-b499-078ce7ddc7e5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:49.021545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 582/1182] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:48 2022] GET /identity/v3/users/e5b59dffce894dbbaafbe1eeff42aabb => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:49.027669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:49.121701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:49.129558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2574ed-fd3b-410d-9009-de95a95a9ea2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:49.164351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e26ea6-967c-47c6-8b23-dd79da7d57ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:49.164957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e26ea6-967c-47c6-8b23-dd79da7d57ac None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:49.165239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e26ea6-967c-47c6-8b23-dd79da7d57ac None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:49.165512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e26ea6-967c-47c6-8b23-dd79da7d57ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:49.234746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e26ea6-967c-47c6-8b23-dd79da7d57ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:49.236208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e26ea6-967c-47c6-8b23-dd79da7d57ac None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:49.328331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:49.336519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 601/1183] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:49.342773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279609e1-cc63-4847-b499-078ce7ddc7e5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:49.389528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:49.443183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279609e1-cc63-4847-b499-078ce7ddc7e5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:49.477228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a5dfd-4740-4e9a-aa34-415213379b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:49.477842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a5dfd-4740-4e9a-aa34-415213379b82 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:49.478146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a5dfd-4740-4e9a-aa34-415213379b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:49.478417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a5dfd-4740-4e9a-aa34-415213379b82 None admin] PATH_INFO: `/v3/projects/c9e41ec835854f24a34a8ed6c3e7520c` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:49.479522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a5dfd-4740-4e9a-aa34-415213379b82 None admin] RBAC: Authorizing `identity:get_project(project_id=c9e41ec835854f24a34a8ed6c3e7520c)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:49.480934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a5dfd-4740-4e9a-aa34-415213379b82 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:49.482593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 583/1184] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:49 2022] GET /identity/v3/projects/c9e41ec835854f24a34a8ed6c3e7520c => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:49.489380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e26ea6-967c-47c6-8b23-dd79da7d57ac None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:49.588520 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e26ea6-967c-47c6-8b23-dd79da7d57ac None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:49.623023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90646c85-63ef-4fe3-b621-c788a26216ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:49.623621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90646c85-63ef-4fe3-b621-c788a26216ef None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:49.623907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90646c85-63ef-4fe3-b621-c788a26216ef None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:49.624177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90646c85-63ef-4fe3-b621-c788a26216ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:49.624895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90646c85-63ef-4fe3-b621-c788a26216ef None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=c9e41ec835854f24a34a8ed6c3e7520c, user.id=e5b59dffce894dbbaafbe1eeff42aabb) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:49.625199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90646c85-63ef-4fe3-b621-c788a26216ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:49.626849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90646c85-63ef-4fe3-b621-c788a26216ef None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:49.632257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 602/1185] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:49 2022] GET /identity/v3/role_assignments?user.id=e5b59dffce894dbbaafbe1eeff42aabb&scope.project.id=c9e41ec835854f24a34a8ed6c3e7520c&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:51.524203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4505c-3215-4cf9-a3a0-13b167d40bf8 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:51.524773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4505c-3215-4cf9-a3a0-13b167d40bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:51.524773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4505c-3215-4cf9-a3a0-13b167d40bf8 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:51.526147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 584/1186] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:51.531603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727e7f9-a811-46bd-b9f6-d9d1e2a0e873 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:51.531995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727e7f9-a811-46bd-b9f6-d9d1e2a0e873 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:51.532166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727e7f9-a811-46bd-b9f6-d9d1e2a0e873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:51.550920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7727e7f9-a811-46bd-b9f6-d9d1e2a0e873 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:51.552262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7727e7f9-a811-46bd-b9f6-d9d1e2a0e873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:51.607087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:51.621092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 603/1187] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:31:51.892667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4505c-3215-4cf9-a3a0-13b167d40bf8 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:52.005166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4505c-3215-4cf9-a3a0-13b167d40bf8 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:52.019977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:52.040274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd35e9a-e5dd-4653-bcca-0d70c28eb85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:52.040886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd35e9a-e5dd-4653-bcca-0d70c28eb85c None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:52.041164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd35e9a-e5dd-4653-bcca-0d70c28eb85c None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:52.041437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd35e9a-e5dd-4653-bcca-0d70c28eb85c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:52.052127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd35e9a-e5dd-4653-bcca-0d70c28eb85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-cbd35e9a-e5dd-4653-bcca-0d70c28eb85c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:52.056763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:52.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:31:52.059001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 585/1188] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:52.064250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727e7f9-a811-46bd-b9f6-d9d1e2a0e873 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:52.163023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727e7f9-a811-46bd-b9f6-d9d1e2a0e873 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:52.195775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:52.196368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:52.196645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:52.196912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:52.197571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:52.197886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:52.199232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:52.207892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 604/1189] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:52.215372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd35e9a-e5dd-4653-bcca-0d70c28eb85c None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:52.317020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd35e9a-e5dd-4653-bcca-0d70c28eb85c None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:52.351057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d83fb4-e2b4-4571-bc10-c3833df4b252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:52.351677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d83fb4-e2b4-4571-bc10-c3833df4b252 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:52.351958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d83fb4-e2b4-4571-bc10-c3833df4b252 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:52.352262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d83fb4-e2b4-4571-bc10-c3833df4b252 None admin] PATH_INFO: `/v3/users/e5b59dffce894dbbaafbe1eeff42aabb` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:52.353386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d83fb4-e2b4-4571-bc10-c3833df4b252 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b59dffce894dbbaafbe1eeff42aabb)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:52.354841 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d83fb4-e2b4-4571-bc10-c3833df4b252 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:52.356424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 586/1190] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:52 2022] GET /identity/v3/users/e5b59dffce894dbbaafbe1eeff42aabb => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:52.362433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:52.412922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:52.464931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2162ab-b6da-46a4-83ad-b8aea2b77a80 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:52.497482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be42729-600d-4207-b18f-0dfa0241a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:52.498123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be42729-600d-4207-b18f-0dfa0241a87c None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:52.498404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be42729-600d-4207-b18f-0dfa0241a87c None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:52.498677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be42729-600d-4207-b18f-0dfa0241a87c None admin] PATH_INFO: `/v3/projects/c9e41ec835854f24a34a8ed6c3e7520c` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:52.499784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be42729-600d-4207-b18f-0dfa0241a87c None admin] RBAC: Authorizing `identity:get_project(project_id=c9e41ec835854f24a34a8ed6c3e7520c)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:52.501880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be42729-600d-4207-b18f-0dfa0241a87c None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:52.503504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 605/1191] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:52 2022] GET /identity/v3/projects/c9e41ec835854f24a34a8ed6c3e7520c => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:52.509632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d83fb4-e2b4-4571-bc10-c3833df4b252 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:52.552889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:52.611169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d83fb4-e2b4-4571-bc10-c3833df4b252 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:52.646213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f4f5d-b832-44bc-ae1a-185c9986c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:52.646862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4f5d-b832-44bc-ae1a-185c9986c39d None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:52.647143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4f5d-b832-44bc-ae1a-185c9986c39d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:52.647418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4f5d-b832-44bc-ae1a-185c9986c39d None admin] PATH_INFO: `/v3/projects/c9e41ec835854f24a34a8ed6c3e7520c/users/e5b59dffce894dbbaafbe1eeff42aabb/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:52.649335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f4f5d-b832-44bc-ae1a-185c9986c39d None admin] RBAC: Authorizing `identity:create_grant(project_id=c9e41ec835854f24a34a8ed6c3e7520c, user_id=e5b59dffce894dbbaafbe1eeff42aabb, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:52.650786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f4f5d-b832-44bc-ae1a-185c9986c39d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:52.747857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 587/1192] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:31:52 2022] PUT /identity/v3/projects/c9e41ec835854f24a34a8ed6c3e7520c/users/e5b59dffce894dbbaafbe1eeff42aabb/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:54.655342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c7618-8461-47e7-bd41-8aad125499a1 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:54.655873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c7618-8461-47e7-bd41-8aad125499a1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:54.655873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c7618-8461-47e7-bd41-8aad125499a1 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:54.657381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 606/1193] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:54.662701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d257f5-aa1e-4139-b542-633367c7160a None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:54.663125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d257f5-aa1e-4139-b542-633367c7160a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:54.663250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d257f5-aa1e-4139-b542-633367c7160a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:54.681482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-77d257f5-aa1e-4139-b542-633367c7160a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:54.682725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-77d257f5-aa1e-4139-b542-633367c7160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:54.749634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 588/1194] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:55.021404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c7618-8461-47e7-bd41-8aad125499a1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:55.123470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c7618-8461-47e7-bd41-8aad125499a1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:55.140677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:55.157495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40f5fc-b8d8-466b-b8ea-d704a538272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:55.158154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40f5fc-b8d8-466b-b8ea-d704a538272b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:55.158439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40f5fc-b8d8-466b-b8ea-d704a538272b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:55.158714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40f5fc-b8d8-466b-b8ea-d704a538272b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:55.169222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40f5fc-b8d8-466b-b8ea-d704a538272b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-aa40f5fc-b8d8-466b-b8ea-d704a538272b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:55.174443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:55.175519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:31:55.176441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 607/1195] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:55.181870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d257f5-aa1e-4139-b542-633367c7160a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:55.212966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:55.281200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d257f5-aa1e-4139-b542-633367c7160a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:55.315860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:55.316462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:55.316745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:55.317022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:55.317724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:55.318059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:55.319429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:55.327421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 589/1196] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:55.333449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40f5fc-b8d8-466b-b8ea-d704a538272b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:55.432619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40f5fc-b8d8-466b-b8ea-d704a538272b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:55.465572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebf48-6940-4cc1-afac-c006a8a01aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:55.466215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebf48-6940-4cc1-afac-c006a8a01aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:55.466495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebf48-6940-4cc1-afac-c006a8a01aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:55.466762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebf48-6940-4cc1-afac-c006a8a01aa1 None admin] PATH_INFO: `/v3/users/e5b59dffce894dbbaafbe1eeff42aabb` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:55.467862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ebf48-6940-4cc1-afac-c006a8a01aa1 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b59dffce894dbbaafbe1eeff42aabb)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:55.470145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ebf48-6940-4cc1-afac-c006a8a01aa1 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:55.471762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 608/1197] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 17:31:55 2022] GET /identity/v3/users/e5b59dffce894dbbaafbe1eeff42aabb => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:55.477675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:55.574461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:55.581906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cb00c-411b-4b49-9214-ab348b154233 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:55.616912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd15ede-7ea3-4576-bf21-ba347184bc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:55.617509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd15ede-7ea3-4576-bf21-ba347184bc03 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:55.617785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd15ede-7ea3-4576-bf21-ba347184bc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:55.618083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd15ede-7ea3-4576-bf21-ba347184bc03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:55.685003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd15ede-7ea3-4576-bf21-ba347184bc03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:55.686439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd15ede-7ea3-4576-bf21-ba347184bc03 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:55.777747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:55.787687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 590/1198] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:55.793832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebf48-6940-4cc1-afac-c006a8a01aa1 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:55.840769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:55.893610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebf48-6940-4cc1-afac-c006a8a01aa1 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:55.928381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f23338-555e-4f83-8184-4644f3146447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:55.929007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f23338-555e-4f83-8184-4644f3146447 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:55.929285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f23338-555e-4f83-8184-4644f3146447 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:55.929551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f23338-555e-4f83-8184-4644f3146447 None admin] PATH_INFO: `/v3/projects/c9e41ec835854f24a34a8ed6c3e7520c` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:55.930687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f23338-555e-4f83-8184-4644f3146447 None admin] RBAC: Authorizing `identity:get_project(project_id=c9e41ec835854f24a34a8ed6c3e7520c)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:55.932103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f23338-555e-4f83-8184-4644f3146447 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:55.933725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 609/1199] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 17:31:55 2022] GET /identity/v3/projects/c9e41ec835854f24a34a8ed6c3e7520c => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:55.941140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd15ede-7ea3-4576-bf21-ba347184bc03 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:56.040405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd15ede-7ea3-4576-bf21-ba347184bc03 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:56.076154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e309ec5-5f8a-4381-b429-608ba054d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:56.076760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e309ec5-5f8a-4381-b429-608ba054d68a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:56.077035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e309ec5-5f8a-4381-b429-608ba054d68a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:56.077310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e309ec5-5f8a-4381-b429-608ba054d68a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:56.078093 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e309ec5-5f8a-4381-b429-608ba054d68a None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=c9e41ec835854f24a34a8ed6c3e7520c, user.id=e5b59dffce894dbbaafbe1eeff42aabb) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:56.079070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e309ec5-5f8a-4381-b429-608ba054d68a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:56.080508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e309ec5-5f8a-4381-b429-608ba054d68a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:56.086405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 591/1200] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:55 2022] GET /identity/v3/role_assignments?user.id=e5b59dffce894dbbaafbe1eeff42aabb&scope.project.id=c9e41ec835854f24a34a8ed6c3e7520c&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:32:05.924203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=43 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:06.083626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=71 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:13.490557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb8130-2576-433f-a8f8-44b50f60e5ae None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:13.491159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb8130-2576-433f-a8f8-44b50f60e5ae None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:13.491159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb8130-2576-433f-a8f8-44b50f60e5ae None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:13.492471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 610/1201] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:13.497569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693a5f-5562-44f7-98bd-cbbd1244125f None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:13.498215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693a5f-5562-44f7-98bd-cbbd1244125f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:13.498503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693a5f-5562-44f7-98bd-cbbd1244125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:13.516632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-52693a5f-5562-44f7-98bd-cbbd1244125f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:13.517837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-52693a5f-5562-44f7-98bd-cbbd1244125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:13.586287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 592/1202] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:33:13.861261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb8130-2576-433f-a8f8-44b50f60e5ae None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:13.974132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb8130-2576-433f-a8f8-44b50f60e5ae None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:14.009159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:14.009788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:14.010103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:14.010380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:14.021703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:14.023197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:14.036982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:14.038269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:14.039671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:14.039671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:14.039671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:14.039671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 611/1203] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:14.044550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52693a5f-5562-44f7-98bd-cbbd1244125f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:14.146747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52693a5f-5562-44f7-98bd-cbbd1244125f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:14.175468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:14.184569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f843757-b4ec-45ff-b8a9-df7cdfd5a3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:14.185180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f843757-b4ec-45ff-b8a9-df7cdfd5a3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:14.186004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f843757-b4ec-45ff-b8a9-df7cdfd5a3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:14.186479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f843757-b4ec-45ff-b8a9-df7cdfd5a3f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:14.187230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f843757-b4ec-45ff-b8a9-df7cdfd5a3f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:14.187534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f843757-b4ec-45ff-b8a9-df7cdfd5a3f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:14.188896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f843757-b4ec-45ff-b8a9-df7cdfd5a3f4 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:14.200699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 593/1204] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:14.207467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:14.260014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:14.309747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb246a4-afe7-4851-ab94-3033dfba45e0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:14.342830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f579d8-fe8d-4220-bb06-c9f5c5a41751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:14.343426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f579d8-fe8d-4220-bb06-c9f5c5a41751 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:14.343752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f579d8-fe8d-4220-bb06-c9f5c5a41751 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:14.344030 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f579d8-fe8d-4220-bb06-c9f5c5a41751 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:14.344662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f579d8-fe8d-4220-bb06-c9f5c5a41751 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:14.346113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f579d8-fe8d-4220-bb06-c9f5c5a41751 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:14.443409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 612/1205] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 17:33:14 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:33:16.390029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05800-cac1-4959-91fd-00b79f43f188 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:16.390567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05800-cac1-4959-91fd-00b79f43f188 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:16.390567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05800-cac1-4959-91fd-00b79f43f188 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:16.391972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 594/1206] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:16.397151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51211-facc-4b1b-aeed-3f286bd5c74b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:16.397597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51211-facc-4b1b-aeed-3f286bd5c74b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:16.397695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51211-facc-4b1b-aeed-3f286bd5c74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:16.416640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c9f51211-facc-4b1b-aeed-3f286bd5c74b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:16.428827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c9f51211-facc-4b1b-aeed-3f286bd5c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:16.486398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 613/1207] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:33:16.756668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05800-cac1-4959-91fd-00b79f43f188 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:16.862263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05800-cac1-4959-91fd-00b79f43f188 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:16.895675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e550b06-07d6-4698-8b28-c5f7c17fee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:16.896294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e550b06-07d6-4698-8b28-c5f7c17fee58 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:16.896577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e550b06-07d6-4698-8b28-c5f7c17fee58 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:16.896851 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e550b06-07d6-4698-8b28-c5f7c17fee58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:16.909485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e550b06-07d6-4698-8b28-c5f7c17fee58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-0e550b06-07d6-4698-8b28-c5f7c17fee58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:33:16.914222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:16.916849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:16.918490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 595/1208] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:16.922493 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51211-facc-4b1b-aeed-3f286bd5c74b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:16.961868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:17.022079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51211-facc-4b1b-aeed-3f286bd5c74b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:17.056480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:17.062078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:17.062078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:17.062078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:17.062078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:17.062078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:17.062078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:17.068399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 614/1209] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:17.074346 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e550b06-07d6-4698-8b28-c5f7c17fee58 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:17.126860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:17.180068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e550b06-07d6-4698-8b28-c5f7c17fee58 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:17.214172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:17.214792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:17.215100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:17.216272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:17.226731 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:17.228220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:17.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:17.246166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:17.247286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:17.247286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:17.247286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:17.247286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 596/1210] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:17.250284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:17.350017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:17.352830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee3f4-1240-4629-a7b9-6523fd23b569 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:17.385498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:17.386124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:17.386407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:17.386678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:17.387318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:17.387618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:17.388967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:17.401133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 615/1211] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:17.407827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:17.510476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796d718-93b9-4c28-bf2c-331547094c09 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:17.512021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:17.545190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:17.545822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:17.546136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:17.546464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:17.554517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:17.556050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:33:17.568009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:17.573255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:17.574272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 597/1212] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:33:17.576226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:17.676948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18449cae-35ca-44d2-95ea-e27286a76903 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:17.712613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:17.713216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:17.713497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:17.713768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:17.714568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:17.714871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:17.716291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:17.733405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 616/1213] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 17:33:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:17.739375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:17.840917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8012f3-934f-4bef-bb87-753847632a3b None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:17.875754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a84cac-c996-4dd2-9c42-376a1a001c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:17.876361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a84cac-c996-4dd2-9c42-376a1a001c54 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:17.876650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a84cac-c996-4dd2-9c42-376a1a001c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:17.876927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a84cac-c996-4dd2-9c42-376a1a001c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:17.907274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:17.950372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a84cac-c996-4dd2-9c42-376a1a001c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:17.950372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a84cac-c996-4dd2-9c42-376a1a001c54 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:18.046799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 598/1214] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:33:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:33:18.052788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:18.072169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:18.153760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6bfe4-1586-4019-a635-f935d5ac02d0 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:18.193762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:18.193762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:18.194784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:18.194929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:18.205323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:18.206794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:18.221497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:18.222656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:18.223683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:18.223683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:18.223683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:18.223683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 617/1215] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:18.228486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a84cac-c996-4dd2-9c42-376a1a001c54 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:18.290278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:18.327818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a84cac-c996-4dd2-9c42-376a1a001c54 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:18.363929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:18.364529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:18.364813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:18.365088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:18.365733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:18.366147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:18.367538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:18.379562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 599/1216] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:18.385430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:18.457160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:18.487182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510c75-a592-45bc-86ff-a52e00f08733 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:18.520681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:18.521338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:18.521621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:18.521926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:18.528770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:18.530685 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:33:18.541989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:18.543073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:18.544091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 618/1217] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:18.549253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:18.654732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35baeb-bd89-4229-9324-7d4c2e65459a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:18.676975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:18.689455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d4918-d939-4e5f-a46d-1ce346f478da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:18.690092 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4918-d939-4e5f-a46d-1ce346f478da None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:18.690373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4918-d939-4e5f-a46d-1ce346f478da None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:18.690647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4918-d939-4e5f-a46d-1ce346f478da None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:18.691341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d4918-d939-4e5f-a46d-1ce346f478da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:18.691643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d4918-d939-4e5f-a46d-1ce346f478da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:18.693100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d4918-d939-4e5f-a46d-1ce346f478da None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:18.705958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 600/1218] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:33:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:18.712314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:18.815397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40b11-40ae-460b-9c4c-1eac04aeb82a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:18.843269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:18.850034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e6bc4-6488-4574-ab5d-3a52e70677cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:18.850637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e6bc4-6488-4574-ab5d-3a52e70677cc None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:18.850918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e6bc4-6488-4574-ab5d-3a52e70677cc None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:18.851190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e6bc4-6488-4574-ab5d-3a52e70677cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:18.851958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e6bc4-6488-4574-ab5d-3a52e70677cc None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=9ea1ee7219994d38a9002b3a1a46383e) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:18.852273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e6bc4-6488-4574-ab5d-3a52e70677cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:18.853733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e6bc4-6488-4574-ab5d-3a52e70677cc None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:18.859014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 619/1219] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:33:18 2022] GET /identity/v3/role_assignments?user.id=9ea1ee7219994d38a9002b3a1a46383e&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:20.767671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9483e2-75c2-43ad-bfc3-f8acd0863b01 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:20.768289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9483e2-75c2-43ad-bfc3-f8acd0863b01 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:20.768289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9483e2-75c2-43ad-bfc3-f8acd0863b01 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:20.769633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 601/1220] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:20.774741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d570b9-7098-4b6c-abe6-928df55ee972 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:20.775162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d570b9-7098-4b6c-abe6-928df55ee972 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:20.775294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d570b9-7098-4b6c-abe6-928df55ee972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:20.795101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-88d570b9-7098-4b6c-abe6-928df55ee972 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:20.796285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-88d570b9-7098-4b6c-abe6-928df55ee972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:20.867038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 620/1221] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:33:21.145451 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9483e2-75c2-43ad-bfc3-f8acd0863b01 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:21.247810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9483e2-75c2-43ad-bfc3-f8acd0863b01 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:21.283508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb996-10ad-4124-834c-450d8a2e48e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:21.284133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb996-10ad-4124-834c-450d8a2e48e2 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:21.284412 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb996-10ad-4124-834c-450d8a2e48e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:21.284684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb996-10ad-4124-834c-450d8a2e48e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:21.295406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacb996-10ad-4124-834c-450d8a2e48e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-eeacb996-10ad-4124-834c-450d8a2e48e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:33:21.300402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:21.303990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:21.304991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 602/1222] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:33:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:21.307854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d570b9-7098-4b6c-abe6-928df55ee972 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:21.407799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d570b9-7098-4b6c-abe6-928df55ee972 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:21.431146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:21.442626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:21.443231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:21.443508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:21.443793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:21.444439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:21.444739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:21.446120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:21.454725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 621/1223] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:33:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:21.460724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb996-10ad-4124-834c-450d8a2e48e2 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:21.507588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:21.565837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb996-10ad-4124-834c-450d8a2e48e2 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:21.601078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:21.601692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:21.601998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:21.602270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:21.613434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:21.615037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:21.629450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:21.630740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:21.631873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:21.631873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:21.631873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:21.631873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 603/1224] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:21.636430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:21.739974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1ebdb-8228-4999-ad46-81735d474b32 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:21.774415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:21.775060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:21.775998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:21.775998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:21.776354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:21.776662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:21.778102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:21.790840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 622/1225] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:21.796687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:21.894593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:21.901809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f197a-08ba-45a3-baeb-92b54bda9984 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:21.938102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:21.938733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:21.939022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:21.939295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:21.946767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:21.948195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:33:21.960047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:21.966099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:21.967258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 604/1226] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:21.968984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:22.017525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:22.072856 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030f9f-9f01-427d-84cb-eff22ade356d None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:22.108614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:22.109271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.109554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.109819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.110576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:22.110880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:22.112292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:22.130361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 623/1227] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 17:33:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:22.136118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:22.242544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697a124-5c3b-47da-b08a-627752267532 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:22.277263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:22.277823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.278139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.278411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.289422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:22.290933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:22.305081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:22.306261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:22.307560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:22.307560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:22.307560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:22.307560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 605/1228] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:22.313280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:22.415243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816adde-520e-4465-993b-5dc57ff13d1a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:22.417607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:22.449730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:22.450461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.450739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.451002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.451633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:22.451995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:22.454316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:22.465260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 624/1229] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:22.471697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:22.481318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:22.575563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36807d75-961e-4e78-9acb-13739b31f42a None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:22.613629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:22.614278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.614554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.614823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.621560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:22.623054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:33:22.635283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:22.636397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:22.637309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 606/1230] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:33:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:22.643265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:22.746934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a90e6-9a61-4263-9ff2-dfac3bd93381 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:22.782194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955364e-f851-49d2-b1d1-315d6cc08257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:22.782858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955364e-f851-49d2-b1d1-315d6cc08257 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.783068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955364e-f851-49d2-b1d1-315d6cc08257 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.783366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955364e-f851-49d2-b1d1-315d6cc08257 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.784068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955364e-f851-49d2-b1d1-315d6cc08257 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:22.784376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955364e-f851-49d2-b1d1-315d6cc08257 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:22.785823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955364e-f851-49d2-b1d1-315d6cc08257 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:22.798669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 625/1231] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:22.804685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:22.871634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:22.906838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f342f1-34af-4301-afda-4e96e4b5fa84 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:22.944479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3311e-601f-47d3-9ac0-5cead6f1e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:22.945137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3311e-601f-47d3-9ac0-5cead6f1e2f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.945419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3311e-601f-47d3-9ac0-5cead6f1e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.945695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3311e-601f-47d3-9ac0-5cead6f1e2f0 None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/9ea1ee7219994d38a9002b3a1a46383e/roles/39294eb951d24b04be9b4b5d98b36640` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.960296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3311e-601f-47d3-9ac0-5cead6f1e2f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=9ea1ee7219994d38a9002b3a1a46383e, role_id=39294eb951d24b04be9b4b5d98b36640)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:22.961733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3311e-601f-47d3-9ac0-5cead6f1e2f0 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:23.122750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 607/1232] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:33:22 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/9ea1ee7219994d38a9002b3a1a46383e/roles/39294eb951d24b04be9b4b5d98b36640 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:33:25.050387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423ac7-1e1c-4f97-837c-face0f6389d8 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:25.050909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423ac7-1e1c-4f97-837c-face0f6389d8 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:25.051023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423ac7-1e1c-4f97-837c-face0f6389d8 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:25.052353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 626/1233] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:25.058122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5cd6-6df2-4b48-bed5-496e43070ea6 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:25.058560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5cd6-6df2-4b48-bed5-496e43070ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:25.058660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5cd6-6df2-4b48-bed5-496e43070ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:25.076995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-331f5cd6-6df2-4b48-bed5-496e43070ea6 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:25.078292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-331f5cd6-6df2-4b48-bed5-496e43070ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:25.145216 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 608/1234] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:33:25.414073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c423ac7-1e1c-4f97-837c-face0f6389d8 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:25.440100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:25.515921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c423ac7-1e1c-4f97-837c-face0f6389d8 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:25.549413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b2e19-e631-4cb8-a0bb-f7759b9055d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:25.550058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b2e19-e631-4cb8-a0bb-f7759b9055d8 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:25.550340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b2e19-e631-4cb8-a0bb-f7759b9055d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:25.550610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b2e19-e631-4cb8-a0bb-f7759b9055d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:25.562482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b2e19-e631-4cb8-a0bb-f7759b9055d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-fb6b2e19-e631-4cb8-a0bb-f7759b9055d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:33:25.567445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:25.568716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:25.570047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 627/1235] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 17:33:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:25.574722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5cd6-6df2-4b48-bed5-496e43070ea6 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:25.633044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:25.688195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5cd6-6df2-4b48-bed5-496e43070ea6 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:25.722326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:25.722916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:25.723196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:25.723465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:25.724121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] RBAC: Adding query filter params (name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:25.724420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:25.725886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:25.733469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 609/1236] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 17:33:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:25.739663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b2e19-e631-4cb8-a0bb-f7759b9055d8 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:25.831415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:25.847959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b2e19-e631-4cb8-a0bb-f7759b9055d8 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:25.881340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:25.882099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:25.882390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:25.882661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:25.895031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:25.896866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:25.911227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:25.912397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:25.913702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:25.913702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:25.913702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:25.913702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 628/1237] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:25.923870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:26.042384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e11bfb-b149-48d2-84a4-229c7fb5b755 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:26.052811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:26.082764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:26.085320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:26.085598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:26.087038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:26.087760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:26.088061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:26.090429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:26.104119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 610/1238] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:33:26.110759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:26.239633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5282496-f3ce-450f-90cd-621df2f08449 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:26.280507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:26.281177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:26.281461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:26.281726 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:26.284489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:26.289704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:26.291199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:33:26.303894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:26.305054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:26.306074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 629/1239] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:26.311701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:26.417251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cf41d-7727-49c3-9c1f-c94935e1f6d3 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:26.454910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:26.455538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:26.455811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:26.456081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:26.456827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:26.456827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:26.458591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:26.478626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 611/1240] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 17:33:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:26.485282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:26.606376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64b1db-06fa-495c-8b9a-92993d6a559b None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:26.645451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6ae9a-323c-431b-b7d4-0d1ca14e0910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:26.646294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ae9a-323c-431b-b7d4-0d1ca14e0910 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:26.646696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ae9a-323c-431b-b7d4-0d1ca14e0910 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:26.647059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ae9a-323c-431b-b7d4-0d1ca14e0910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:26.715844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:26.722511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6ae9a-323c-431b-b7d4-0d1ca14e0910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:26.723974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6ae9a-323c-431b-b7d4-0d1ca14e0910 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:26.824269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 630/1241] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:33:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:33:26.831034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:26.860082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:26.934270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc23b7e-2dea-428f-b822-ffddc6df1df9 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:26.970051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:26.972040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:26.972040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:26.972040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:26.981539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:26.983052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:26.999360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:27.006685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:27.007629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:27.007629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:27.007629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:27.007629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 612/1242] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:27.007875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6ae9a-323c-431b-b7d4-0d1ca14e0910 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:27.110053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:27.118168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6ae9a-323c-431b-b7d4-0d1ca14e0910 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:27.151794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:27.152399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:27.152681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:27.152953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:27.153600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:27.153938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:27.155304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:27.166307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 631/1243] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:27.172967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:27.253488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:27.274739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27122abe-6fa7-4b8c-b61d-39c68e8191c7 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:27.308462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:27.309076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:27.309359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:27.309688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:27.316964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:27.318426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:33:27.330438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:27.333275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:27.334508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 613/1244] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:27.337736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:27.440188 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef33e33-7ef7-415a-bd40-6d9e305b335f None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:27.473476 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164eb8c-9dd8-4ce8-b03e-26daaf17d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:27.474102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164eb8c-9dd8-4ce8-b03e-26daaf17d84a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:27.474386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164eb8c-9dd8-4ce8-b03e-26daaf17d84a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:27.474659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164eb8c-9dd8-4ce8-b03e-26daaf17d84a None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:27.475352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164eb8c-9dd8-4ce8-b03e-26daaf17d84a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:27.475658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164eb8c-9dd8-4ce8-b03e-26daaf17d84a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:27.477116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164eb8c-9dd8-4ce8-b03e-26daaf17d84a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:27.489766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 632/1245] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:27.495806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:27.599774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f1c16-a1fb-4d6a-a3f1-5e2293230b24 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:27.633789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc449b-3af2-4ad7-a58e-0b89c7af4ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:27.634418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc449b-3af2-4ad7-a58e-0b89c7af4ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:27.634700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc449b-3af2-4ad7-a58e-0b89c7af4ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:27.634977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc449b-3af2-4ad7-a58e-0b89c7af4ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:27.635779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc449b-3af2-4ad7-a58e-0b89c7af4ea5 None admin] RBAC: Adding query filter params (role.id=39294eb951d24b04be9b4b5d98b36640, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=9ea1ee7219994d38a9002b3a1a46383e) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:27.636024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc449b-3af2-4ad7-a58e-0b89c7af4ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:27.637441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc449b-3af2-4ad7-a58e-0b89c7af4ea5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:27.643464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 614/1246] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:33:27 2022] GET /identity/v3/role_assignments?user.id=9ea1ee7219994d38a9002b3a1a46383e&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=39294eb951d24b04be9b4b5d98b36640 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:29.658663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a715d-9ea5-4bd2-8ee8-c47a03858d96 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:29.659193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a715d-9ea5-4bd2-8ee8-c47a03858d96 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:29.659193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a715d-9ea5-4bd2-8ee8-c47a03858d96 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:29.660554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 633/1247] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:29.666095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781890-47ca-4be2-addd-490b463574e7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:29.667414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781890-47ca-4be2-addd-490b463574e7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:29.667414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781890-47ca-4be2-addd-490b463574e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:29.673961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:29.686251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2d781890-47ca-4be2-addd-490b463574e7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:29.688018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2d781890-47ca-4be2-addd-490b463574e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:29.755464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 615/1248] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:33:30.031986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a715d-9ea5-4bd2-8ee8-c47a03858d96 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:30.050568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:30.136194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a715d-9ea5-4bd2-8ee8-c47a03858d96 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:30.172255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa903f25-29d4-4f5c-8019-77f7c8c04f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:30.172832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa903f25-29d4-4f5c-8019-77f7c8c04f74 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.173115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa903f25-29d4-4f5c-8019-77f7c8c04f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.173394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa903f25-29d4-4f5c-8019-77f7c8c04f74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.183746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa903f25-29d4-4f5c-8019-77f7c8c04f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-fa903f25-29d4-4f5c-8019-77f7c8c04f74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:33:30.188390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:30.189537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:30.190606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 634/1249] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:30.195617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d781890-47ca-4be2-addd-490b463574e7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:30.296479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d781890-47ca-4be2-addd-490b463574e7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:30.325527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:30.331177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:30.331831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.332107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.332377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.333043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:30.333344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:30.334727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:30.342716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 616/1250] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:33:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:30.349337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa903f25-29d4-4f5c-8019-77f7c8c04f74 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:30.427764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:30.450624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa903f25-29d4-4f5c-8019-77f7c8c04f74 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:30.487840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:30.488474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.488752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.489021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.499425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:30.501067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:30.515737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:30.517494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:30.518426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:30.518426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:30.518426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:30.518426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 635/1251] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:30.523614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:30.627151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865d885-1682-46dc-b7de-cbd32462e878 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:30.661678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:30.662383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.662668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.662957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.663671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:30.663978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:30.665349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:30.677248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 617/1252] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:30.683360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:30.784893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660e958-f1d5-4788-8553-191d0ca5c0a3 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:30.810908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:30.819136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:30.819760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.820039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.820309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.829747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:30.831838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:33:30.843136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:30.844289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:30.845494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 636/1253] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:30.850209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:30.909265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:30.954693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52620e-5aa9-44e5-8f8f-5af89fc21ebb None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:30.989572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:30.990206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.990489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.990763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.991497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:30.991801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:30.993212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:31.011937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 618/1254] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 17:33:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:31.018827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:31.120764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9724b2-dcff-4357-8981-20af4313ef23 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:31.156365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95935d8-bf87-4239-a9c1-d12eaaed6d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:31.157688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95935d8-bf87-4239-a9c1-d12eaaed6d42 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:31.158000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95935d8-bf87-4239-a9c1-d12eaaed6d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:31.158273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95935d8-bf87-4239-a9c1-d12eaaed6d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:31.216339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:31.229078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95935d8-bf87-4239-a9c1-d12eaaed6d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:31.230559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95935d8-bf87-4239-a9c1-d12eaaed6d42 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:31.327857 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 637/1255] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:33:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:33:31.334676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:31.436791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cd563-3e19-498a-825a-bafd415c35e5 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:31.450227 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:31.472782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:31.473343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:31.473626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:31.473930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:31.483859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:31.485325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:31.499418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:31.501398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:31.502863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:31.502863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:31.502863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:31.502863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 619/1256] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:31.506870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95935d8-bf87-4239-a9c1-d12eaaed6d42 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:31.599912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:31.612590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95935d8-bf87-4239-a9c1-d12eaaed6d42 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:31.646910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:31.647529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:31.647788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:31.648056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:31.648717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:31.649015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:31.650397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:31.661919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 638/1257] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:31.667701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:31.771646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753fd81-2cc9-4e0d-b1de-3de8ada8ba5f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:31.804765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:31.805387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:31.805702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:31.806007 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:31.813618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:31.815087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:33:31.826333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:31.828295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:31.830210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 620/1258] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:31.834221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:31.937798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ed1f3-50ef-4c25-8169-b42678c5b56e None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:31.971979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3f548-3b4e-4f50-b57c-593709c47678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:31.972586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f548-3b4e-4f50-b57c-593709c47678 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:31.972864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f548-3b4e-4f50-b57c-593709c47678 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:31.973136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f548-3b4e-4f50-b57c-593709c47678 None admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:31.973824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3f548-3b4e-4f50-b57c-593709c47678 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:31.974161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3f548-3b4e-4f50-b57c-593709c47678 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:31.975580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3f548-3b4e-4f50-b57c-593709c47678 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:31.987855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 639/1259] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:31.994168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:32.021075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:32.096947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9919a0-ce09-4969-859b-6c5074748d01 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:32.132232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f8a66-38d9-4474-ad3c-dbf797837afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:32.132834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8a66-38d9-4474-ad3c-dbf797837afa None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:32.133119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8a66-38d9-4474-ad3c-dbf797837afa None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:32.133392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8a66-38d9-4474-ad3c-dbf797837afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:32.134157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f8a66-38d9-4474-ad3c-dbf797837afa None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=9ea1ee7219994d38a9002b3a1a46383e) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:32.134467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f8a66-38d9-4474-ad3c-dbf797837afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:32.135918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f8a66-38d9-4474-ad3c-dbf797837afa None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:32.141722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 621/1260] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:33:31 2022] GET /identity/v3/role_assignments?user.id=9ea1ee7219994d38a9002b3a1a46383e&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:34.075613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b60771-4112-4835-ad4e-b080d2462b5f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:34.076155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b60771-4112-4835-ad4e-b080d2462b5f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:34.076155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b60771-4112-4835-ad4e-b080d2462b5f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:34.077536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 640/1261] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:33:34.083645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5901a-b035-4e80-ab6e-eac87c498927 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:34.084065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5901a-b035-4e80-ab6e-eac87c498927 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:34.084222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5901a-b035-4e80-ab6e-eac87c498927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:34.103515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6fb5901a-b035-4e80-ab6e-eac87c498927 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:34.104729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6fb5901a-b035-4e80-ab6e-eac87c498927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:34.171785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 622/1262] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:33:34.445476 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b60771-4112-4835-ad4e-b080d2462b5f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:34.457377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:34.547409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b60771-4112-4835-ad4e-b080d2462b5f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:34.583214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d95166-aca7-4f2b-934f-4abc17586154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:34.583846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d95166-aca7-4f2b-934f-4abc17586154 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:34.592424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d95166-aca7-4f2b-934f-4abc17586154 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:34.592424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d95166-aca7-4f2b-934f-4abc17586154 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:34.595092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d95166-aca7-4f2b-934f-4abc17586154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-d1d95166-aca7-4f2b-934f-4abc17586154 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:33:34.599895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:34.601013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:34.602277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 641/1263] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:33:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:34.607372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5901a-b035-4e80-ab6e-eac87c498927 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:34.626432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:34.708780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5901a-b035-4e80-ab6e-eac87c498927 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:34.743617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:34.744218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:34.744499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:34.744766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:34.745423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:34.745723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:34.747113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:34.755603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 623/1264] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:33:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:34.761515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d95166-aca7-4f2b-934f-4abc17586154 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:34.830738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:34.864602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d95166-aca7-4f2b-934f-4abc17586154 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:34.898601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:34.899230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:34.899509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:34.899780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:34.911165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:34.912606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:34.927949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:34.932883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:34.934059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:34.934059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:34.934059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:34.934059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 642/1265] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:34.935285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:35.021844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:35.040849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c04610-43f2-40a9-96cb-e21f25ef898f None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:35.075827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:35.076425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:35.076708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:35.076981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:35.077620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:35.077946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:35.079352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:35.090928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 624/1266] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:35.097049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:35.198029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f353ea8-0d08-498e-b727-4e7d2d78d4c5 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:35.218872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:35.232772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:35.233399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:35.233682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:35.240058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:35.243107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:35.244563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:33:35.256128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:35.257161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:35.263713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 643/1267] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:35.264397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:35.366312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a42f4-a941-4bdf-9cce-79c65880aa79 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:35.399810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:35.400408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:35.400681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:35.400949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:35.401664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:35.401991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:35.403396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:35.412906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:35.421111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 625/1268] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 17:33:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:35.427592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:35.532852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2c282-3da9-4e8d-a77d-2afcbc10b1ce None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:35.566863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:35.567491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:35.567756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:35.568022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:35.580275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:35.581776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:35.596717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:35.599309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:35.601087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:35.601087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:35.601087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:35.602120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 644/1269] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:35.603765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:35.707358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82085414-65bf-418c-98ab-596aed879a4d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:35.742019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:35.742616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:35.742894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:35.743162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:35.743803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:35.744129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:35.745492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:35.758425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 626/1270] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:35.764193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:35.795954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:35.864688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f91aa-e0c0-44de-954f-6e28ff9806ac None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:35.901056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:35.901691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:35.902048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:35.902408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:35.909577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:35.913801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:33:35.924112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:35.926155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:35.928273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 645/1271] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:35.931551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:35.994673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:36.037210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4fbf5-b64c-4fd5-ad73-6df1b5971165 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:36.072829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95f2d9-f21a-4255-a128-47ca0d6a9967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:36.073426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95f2d9-f21a-4255-a128-47ca0d6a9967 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:36.073842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95f2d9-f21a-4255-a128-47ca0d6a9967 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:36.074151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95f2d9-f21a-4255-a128-47ca0d6a9967 None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:36.074843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95f2d9-f21a-4255-a128-47ca0d6a9967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:36.075155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95f2d9-f21a-4255-a128-47ca0d6a9967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:36.076584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95f2d9-f21a-4255-a128-47ca0d6a9967 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:36.089176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 627/1272] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:36.095801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:36.192613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:36.201152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de4ac4-94d4-445f-a594-f0279db28412 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:36.234419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de04896-e4d3-449c-ab55-c7f4879ee39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:36.235349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04896-e4d3-449c-ab55-c7f4879ee39e None admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:36.235623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04896-e4d3-449c-ab55-c7f4879ee39e None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:36.235891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04896-e4d3-449c-ab55-c7f4879ee39e None admin] PATH_INFO: `/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/9ea1ee7219994d38a9002b3a1a46383e/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:36.237769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de04896-e4d3-449c-ab55-c7f4879ee39e None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc1eb9572154f2e8de63401b760a1c9, user_id=9ea1ee7219994d38a9002b3a1a46383e, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:36.239304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de04896-e4d3-449c-ab55-c7f4879ee39e None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:36.356063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 646/1273] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 17:33:36 2022] PUT /identity/v3/projects/6dc1eb9572154f2e8de63401b760a1c9/users/9ea1ee7219994d38a9002b3a1a46383e/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:33:38.269292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419285ca-ac18-4d99-8593-dcd9fd5df4be None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:38.269816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419285ca-ac18-4d99-8593-dcd9fd5df4be None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:38.269952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419285ca-ac18-4d99-8593-dcd9fd5df4be None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:38.271337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 628/1274] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:38.276524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc305bd-56e6-4979-a1ff-e0d1dff09c6f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:38.276942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc305bd-56e6-4979-a1ff-e0d1dff09c6f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:38.277092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc305bd-56e6-4979-a1ff-e0d1dff09c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:38.296869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-5bc305bd-56e6-4979-a1ff-e0d1dff09c6f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:38.298319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5bc305bd-56e6-4979-a1ff-e0d1dff09c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:38.365743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 647/1275] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:33:38.640654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419285ca-ac18-4d99-8593-dcd9fd5df4be None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:38.743502 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419285ca-ac18-4d99-8593-dcd9fd5df4be None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:38.761137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:38.777987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cc0c-2220-4868-9c7b-6cd7b9853ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:38.778608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cc0c-2220-4868-9c7b-6cd7b9853ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:38.778886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cc0c-2220-4868-9c7b-6cd7b9853ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:38.779154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cc0c-2220-4868-9c7b-6cd7b9853ee1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:38.789099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0cc0c-2220-4868-9c7b-6cd7b9853ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-1fb0cc0c-2220-4868-9c7b-6cd7b9853ee1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:33:38.793794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:38.798565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:38.799612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 629/1276] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 17:33:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:38.802278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc305bd-56e6-4979-a1ff-e0d1dff09c6f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:38.885898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:38.903589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc305bd-56e6-4979-a1ff-e0d1dff09c6f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:38.940089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:38.940699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:38.940980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:38.941249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:38.941947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:38.942259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:38.943664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:38.951813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 648/1277] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 17:33:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:38.958023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cc0c-2220-4868-9c7b-6cd7b9853ee1 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:39.060260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cc0c-2220-4868-9c7b-6cd7b9853ee1 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:39.093316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:39.094571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:39.094850 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:39.095115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:39.105315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:39.107522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:39.120801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:39.121961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:39.122884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:39.122884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:39.122884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:39.122884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 630/1278] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:39.128217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:39.229082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a66a25-708b-49af-9f02-d658acd889d2 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:39.262605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:39.263237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:39.263516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:39.263788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:39.264432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:39.264735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:39.266817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:39.278425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 649/1279] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:39.284268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:39.316239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:39.385713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418343e-28be-4381-bb93-0f75296a6bba None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:39.419008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:39.419637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:39.419915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:39.420181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:39.428856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:39.430452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:33:39.442388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:39.443588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:39.444806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 631/1280] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:39.449448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:39.453332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:39.553078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f52fc-1022-4d97-942a-8e382d0837a7 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:39.590416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:39.591023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:39.592111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:39.592111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:39.592309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:39.592573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:39.594022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:39.612972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 650/1281] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 17:33:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:39.618985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:39.720174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:39.729870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf44ae-7980-4063-b9f2-28c0aae7792d None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:39.764201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cd0ea-44e4-4362-870b-1523e032f8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:39.764801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cd0ea-44e4-4362-870b-1523e032f8ba None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:39.765078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cd0ea-44e4-4362-870b-1523e032f8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:39.765350 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cd0ea-44e4-4362-870b-1523e032f8ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:39.832477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cd0ea-44e4-4362-870b-1523e032f8ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:39.833832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cd0ea-44e4-4362-870b-1523e032f8ba None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:39.928063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:39.938822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 632/1282] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 17:33:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:33:39.944808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:40.008286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:40.046941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1313f-e12d-4ce5-b2c5-65720c7892b4 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:40.082307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:40.086961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:40.086961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:40.086961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:40.093313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:40.095415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:33:40.109445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:40.121302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:33:40.122460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:33:40.122460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:33:40.122460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:40.122460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 651/1283] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:40.122751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cd0ea-44e4-4362-870b-1523e032f8ba None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:40.216571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cd0ea-44e4-4362-870b-1523e032f8ba None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:40.250616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:40.251217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:40.251523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:40.251807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:40.252454 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:40.252758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:40.254144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:40.265826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 633/1284] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 17:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:33:40.272333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:40.374626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd12cf-6dbd-467f-b64d-5b44b848b998 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:40.387914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:40.409563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:40.410340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:40.410737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:40.411116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:40.418098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:40.419531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:33:40.431016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:40.432094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: ERROR keystone.server.flask.application Aug 24 17:33:40.433210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 652/1285] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 17:33:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:40.438447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:40.489699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:40.541136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26794699-dd31-48ac-8312-0cb38c06b60e None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:40.575545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb24e87-22e9-4f0a-882a-391eb9cceb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:40.576157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24e87-22e9-4f0a-882a-391eb9cceb4b None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:40.576435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24e87-22e9-4f0a-882a-391eb9cceb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:40.576702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24e87-22e9-4f0a-882a-391eb9cceb4b None admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:40.577465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb24e87-22e9-4f0a-882a-391eb9cceb4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:40.577827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb24e87-22e9-4f0a-882a-391eb9cceb4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:40.580245 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb24e87-22e9-4f0a-882a-391eb9cceb4b None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:40.592763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 634/1286] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 17:33:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:40.600664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:40.704462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50760777-96a2-49b5-b55e-64f18bfa401a None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:40.737973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8546d9e-2223-44e1-bb45-6456438e4ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:40.738566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8546d9e-2223-44e1-bb45-6456438e4ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:40.738844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8546d9e-2223-44e1-bb45-6456438e4ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:40.739114 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8546d9e-2223-44e1-bb45-6456438e4ed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:40.739831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8546d9e-2223-44e1-bb45-6456438e4ed5 None admin] RBAC: Adding query filter params (role.id=7cea93ff9a6d44b2b8f0f6b15518fee1, scope.project.id=6dc1eb9572154f2e8de63401b760a1c9, user.id=9ea1ee7219994d38a9002b3a1a46383e) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:40.740135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8546d9e-2223-44e1-bb45-6456438e4ed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:40.741543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8546d9e-2223-44e1-bb45-6456438e4ed5 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:40.747202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 653/1287] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 17:33:40 2022] GET /identity/v3/role_assignments?user.id=9ea1ee7219994d38a9002b3a1a46383e&scope.project.id=6dc1eb9572154f2e8de63401b760a1c9&role.id=7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:42.682383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9bb9-985b-43c9-b699-a6621e51e7e5 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:42.682980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9bb9-985b-43c9-b699-a6621e51e7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:42.682980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9bb9-985b-43c9-b699-a6621e51e7e5 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:42.684292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 635/1288] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:42.689757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b4dd0-90fd-4380-88b6-45c6826d90b5 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:42.691166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b4dd0-90fd-4380-88b6-45c6826d90b5 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:42.691166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b4dd0-90fd-4380-88b6-45c6826d90b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:42.709072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-788b4dd0-90fd-4380-88b6-45c6826d90b5 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:42.710355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-788b4dd0-90fd-4380-88b6-45c6826d90b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:42.740015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:42.777638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 654/1289] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:33:43.051155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9bb9-985b-43c9-b699-a6621e51e7e5 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:43.159031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9bb9-985b-43c9-b699-a6621e51e7e5 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:43.170206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:43.195100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:43.199328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:43.199328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:43.199328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:43.199328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:43.199328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:43.207106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:43.208191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:43.209416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 636/1290] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:33:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:43.214240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b4dd0-90fd-4380-88b6-45c6826d90b5 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:43.313500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b4dd0-90fd-4380-88b6-45c6826d90b5 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:43.347710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d9514-5aa8-4a0b-988f-8245a5cfdb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:43.348306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9514-5aa8-4a0b-988f-8245a5cfdb92 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:43.348649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9514-5aa8-4a0b-988f-8245a5cfdb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:43.348915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9514-5aa8-4a0b-988f-8245a5cfdb92 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:43.349541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d9514-5aa8-4a0b-988f-8245a5cfdb92 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:43.349843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d9514-5aa8-4a0b-988f-8245a5cfdb92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:43.351281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d9514-5aa8-4a0b-988f-8245a5cfdb92 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:43.355998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 655/1291] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 24 17:33:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:43.362473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:43.467941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d71279-77c4-4fd7-95ec-0fc2065e69fe None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:43.502954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47b9b0-b727-493e-b7b7-3d0f1ccb2ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:43.503549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47b9b0-b727-493e-b7b7-3d0f1ccb2ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:43.503830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47b9b0-b727-493e-b7b7-3d0f1ccb2ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:43.504103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47b9b0-b727-493e-b7b7-3d0f1ccb2ed6 None admin] PATH_INFO: `/v3/services` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:43.504729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47b9b0-b727-493e-b7b7-3d0f1ccb2ed6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:43.505034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47b9b0-b727-493e-b7b7-3d0f1ccb2ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:43.506439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47b9b0-b727-493e-b7b7-3d0f1ccb2ed6 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:43.511044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 637/1292] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 24 17:33:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:45.441908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76aa1a-3899-487f-b846-97e83fe5786f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:45.442480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76aa1a-3899-487f-b846-97e83fe5786f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:45.442480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76aa1a-3899-487f-b846-97e83fe5786f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:45.443790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 656/1293] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:45.449060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66edaa-9da0-485f-b301-f9b638c9567e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:45.449447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66edaa-9da0-485f-b301-f9b638c9567e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:45.449620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66edaa-9da0-485f-b301-f9b638c9567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:45.468553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ce66edaa-9da0-485f-b301-f9b638c9567e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:45.470469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ce66edaa-9da0-485f-b301-f9b638c9567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:45.502091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:45.540290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 638/1294] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:33:45.813214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76aa1a-3899-487f-b846-97e83fe5786f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:45.845696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:45.917673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76aa1a-3899-487f-b846-97e83fe5786f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:45.956966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc5068-5c5e-4d6d-a4e3-db64bac7cab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:45.957574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc5068-5c5e-4d6d-a4e3-db64bac7cab0 None admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:45.957887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc5068-5c5e-4d6d-a4e3-db64bac7cab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:45.958154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc5068-5c5e-4d6d-a4e3-db64bac7cab0 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:45.958813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc5068-5c5e-4d6d-a4e3-db64bac7cab0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:45.960225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc5068-5c5e-4d6d-a4e3-db64bac7cab0 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:46.083131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 657/1295] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 17:33:45 2022] POST /identity/v3/services => generated 246 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:33:47.994644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d0e51-3942-426e-b718-2983fbec51e2 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:47.995223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d0e51-3942-426e-b718-2983fbec51e2 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:47.995223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d0e51-3942-426e-b718-2983fbec51e2 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:47.996604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 639/1296] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:48.002519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee1ae2-32b7-4c61-8776-26d0d66f29e6 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.002962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee1ae2-32b7-4c61-8776-26d0d66f29e6 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.003065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee1ae2-32b7-4c61-8776-26d0d66f29e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.022001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4aee1ae2-32b7-4c61-8776-26d0d66f29e6 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:48.023321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4aee1ae2-32b7-4c61-8776-26d0d66f29e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:48.100149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 658/1297] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:33:48.372248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d0e51-3942-426e-b718-2983fbec51e2 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:48.488890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d0e51-3942-426e-b718-2983fbec51e2 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:48.526397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:48.527042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.527321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.527596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.528185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:48.529576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:48.539675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:48.540712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:48.541579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 640/1298] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:33:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:33:48.547452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee1ae2-32b7-4c61-8776-26d0d66f29e6 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:48.568398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:48.653597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee1ae2-32b7-4c61-8776-26d0d66f29e6 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:48.690495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:48.691115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.691436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.691690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.692330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:48.692629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:48.694018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:48.699319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 659/1299] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 24 17:33:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:48.706306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:48.738901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:48.810776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfc028-4cbb-40fd-b7bc-f96291d0aa14 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:48.845817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163643e-528f-4f7a-b37a-69d874115ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:48.846442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163643e-528f-4f7a-b37a-69d874115ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.846720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163643e-528f-4f7a-b37a-69d874115ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.846988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163643e-528f-4f7a-b37a-69d874115ce8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.847691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163643e-528f-4f7a-b37a-69d874115ce8 None admin] RBAC: Adding query filter params (interface=public, service_id=f146e9946e814950bed7a23bbe165bf1, region_id=RegionOne) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:48.847998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163643e-528f-4f7a-b37a-69d874115ce8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:48.849373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163643e-528f-4f7a-b37a-69d874115ce8 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:48.854802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 641/1300] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 17:33:48 2022] GET /identity/v3/endpoints?service_id=f146e9946e814950bed7a23bbe165bf1&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:48.860970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:48.945321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:48.963576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7bbf-de5d-4ffe-a1d1-94dfe219f7f6 None admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:48.997221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36be80-fe62-4621-b15e-5dfad4dd799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:48.997823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36be80-fe62-4621-b15e-5dfad4dd799a None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.998145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36be80-fe62-4621-b15e-5dfad4dd799a None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.998421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36be80-fe62-4621-b15e-5dfad4dd799a None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.999031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36be80-fe62-4621-b15e-5dfad4dd799a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:49.000961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36be80-fe62-4621-b15e-5dfad4dd799a None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:49.007942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 660/1301] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 17:33:48 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:33:50.917619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86260fb1-1f1d-464a-b889-ccb7862f5008 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:50.918030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86260fb1-1f1d-464a-b889-ccb7862f5008 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:50.918206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86260fb1-1f1d-464a-b889-ccb7862f5008 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:50.919544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 642/1302] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:50.925138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4db8-54db-47e0-91cd-2e3fa4193f5f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:50.925616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4db8-54db-47e0-91cd-2e3fa4193f5f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:50.925760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4db8-54db-47e0-91cd-2e3fa4193f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:50.945782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-811a4db8-54db-47e0-91cd-2e3fa4193f5f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:50.946989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-811a4db8-54db-47e0-91cd-2e3fa4193f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:51.014273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 661/1303] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:33:51.287851 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86260fb1-1f1d-464a-b889-ccb7862f5008 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:51.388957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:51.396003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86260fb1-1f1d-464a-b889-ccb7862f5008 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:51.430971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:51.431606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:51.431887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:51.432161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:51.432744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:51.434139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:33:51.443598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:33:51.446361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application with Lock( Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: ERROR keystone.server.flask.application Aug 24 17:33:51.447382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 643/1304] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 17:33:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:33:51.450809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a4db8-54db-47e0-91cd-2e3fa4193f5f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:51.530271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:51.553331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a4db8-54db-47e0-91cd-2e3fa4193f5f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:51.592459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5c806-b2f7-4ebe-8d34-4825c4c530b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:51.593062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5c806-b2f7-4ebe-8d34-4825c4c530b3 None admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:51.593341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5c806-b2f7-4ebe-8d34-4825c4c530b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:51.593611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5c806-b2f7-4ebe-8d34-4825c4c530b3 None admin] PATH_INFO: `/v3/services` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:51.594568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5c806-b2f7-4ebe-8d34-4825c4c530b3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:33:51.594865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5c806-b2f7-4ebe-8d34-4825c4c530b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:51.596220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5c806-b2f7-4ebe-8d34-4825c4c530b3 None admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:51.601264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 662/1305] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 24 17:33:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:33:51.607268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:51.708267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72ee9a-4d38-44a0-a1f6-c35de86d0300 None admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:51.742155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c98835-c468-4a88-9b2d-87cf1e1a2ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:51.742756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c98835-c468-4a88-9b2d-87cf1e1a2ae7 None admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:51.743085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c98835-c468-4a88-9b2d-87cf1e1a2ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:51.743358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c98835-c468-4a88-9b2d-87cf1e1a2ae7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:51.744010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c98835-c468-4a88-9b2d-87cf1e1a2ae7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:51.745368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c98835-c468-4a88-9b2d-87cf1e1a2ae7 None admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:51.848116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 644/1306] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:33:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:01.598277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=30 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:01.842666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:50.970123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b987215-6dac-4bab-9c2b-ca1c6a47dea0 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.970995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b987215-6dac-4bab-9c2b-ca1c6a47dea0 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.971147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b987215-6dac-4bab-9c2b-ca1c6a47dea0 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.979911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c131a-db45-42dc-828f-6dddee8ab200 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.980573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c131a-db45-42dc-828f-6dddee8ab200 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.980999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c131a-db45-42dc-828f-6dddee8ab200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.982928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 663/1307] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:40:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:51.066811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9b1c131a-db45-42dc-828f-6dddee8ab200 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:51.068180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9b1c131a-db45-42dc-828f-6dddee8ab200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:51.198224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 645/1308] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:51.497134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bbdea-c4a7-44ed-a803-e4ffd5a050dd None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.497414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bbdea-c4a7-44ed-a803-e4ffd5a050dd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.497937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bbdea-c4a7-44ed-a803-e4ffd5a050dd None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.499864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 664/1309] 10.222.0.76 () {54 vars in 965 bytes} [Wed Aug 24 17:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:40:51.507067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd19996-eae0-46fe-ad69-3acd700b5e88 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.507450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd19996-eae0-46fe-ad69-3acd700b5e88 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.507803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd19996-eae0-46fe-ad69-3acd700b5e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.595330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0cd19996-eae0-46fe-ad69-3acd700b5e88 None None] MFA Rules not processed for user `efc068011c14492d9ba483650cf632e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:51.596660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0cd19996-eae0-46fe-ad69-3acd700b5e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:51.618956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:51.680615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 646/1310] 10.222.0.76 () {58 vars in 1062 bytes} [Wed Aug 24 17:40:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:51.691522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bbdea-c4a7-44ed-a803-e4ffd5a050dd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.693567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-eb5bbdea-c4a7-44ed-a803-e4ffd5a050dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:51.827967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bbdea-c4a7-44ed-a803-e4ffd5a050dd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.917629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd05630-da5c-4f59-8469-24ce8c6ebee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.918304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd05630-da5c-4f59-8469-24ce8c6ebee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.918590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd05630-da5c-4f59-8469-24ce8c6ebee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.918867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd05630-da5c-4f59-8469-24ce8c6ebee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.920574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ebd05630-da5c-4f59-8469-24ce8c6ebee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:51.982995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:52.041272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd05630-da5c-4f59-8469-24ce8c6ebee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:52.042740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd05630-da5c-4f59-8469-24ce8c6ebee8 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:52.063623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 665/1311] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:54.302177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f746-c045-4f14-b39c-195a743bbd6a None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:54.303148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f746-c045-4f14-b39c-195a743bbd6a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:54.303148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f746-c045-4f14-b39c-195a743bbd6a None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:54.304368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 647/1312] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:54.310742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474b3c1-5247-4e4d-90d4-e1af45fcb196 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:54.311346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474b3c1-5247-4e4d-90d4-e1af45fcb196 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:54.311522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474b3c1-5247-4e4d-90d4-e1af45fcb196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:54.409925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-6474b3c1-5247-4e4d-90d4-e1af45fcb196 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:54.411357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6474b3c1-5247-4e4d-90d4-e1af45fcb196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:54.525906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 666/1313] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:54.825194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55f12-7a50-4c45-abda-56fb3ed2beae None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:54.825484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55f12-7a50-4c45-abda-56fb3ed2beae None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:54.825769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55f12-7a50-4c45-abda-56fb3ed2beae None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:54.827226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 648/1314] 10.222.0.76 () {54 vars in 965 bytes} [Wed Aug 24 17:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:54.833704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b1414-f806-4147-af66-9850ed4d30a5 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:54.834024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b1414-f806-4147-af66-9850ed4d30a5 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:54.834326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b1414-f806-4147-af66-9850ed4d30a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:54.846478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:54.915885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7d7b1414-f806-4147-af66-9850ed4d30a5 None None] MFA Rules not processed for user `efc068011c14492d9ba483650cf632e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:54.917559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7d7b1414-f806-4147-af66-9850ed4d30a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:54.999963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 667/1315] 10.222.0.76 () {58 vars in 1062 bytes} [Wed Aug 24 17:40:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:55.009125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55f12-7a50-4c45-abda-56fb3ed2beae None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:55.010713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a2f55f12-7a50-4c45-abda-56fb3ed2beae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:55.122398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55f12-7a50-4c45-abda-56fb3ed2beae None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:55.197552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:55.200024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588269d-9e3e-4de4-9747-9252af1b9c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:55.200637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588269d-9e3e-4de4-9747-9252af1b9c9b service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:55.200919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588269d-9e3e-4de4-9747-9252af1b9c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:55.201193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588269d-9e3e-4de4-9747-9252af1b9c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:55.203048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a588269d-9e3e-4de4-9747-9252af1b9c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:55.326493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588269d-9e3e-4de4-9747-9252af1b9c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:55.328178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588269d-9e3e-4de4-9747-9252af1b9c9b service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:55.350991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 649/1316] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:58.041275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec18895-6916-4cae-9508-e3e4d4b7e63f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:58.041802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec18895-6916-4cae-9508-e3e4d4b7e63f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:58.041972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec18895-6916-4cae-9508-e3e4d4b7e63f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:58.043222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 668/1317] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:58.049778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498886-defd-4cc2-a9fb-fc4fab8e7fd4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:58.050249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498886-defd-4cc2-a9fb-fc4fab8e7fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:58.050343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498886-defd-4cc2-a9fb-fc4fab8e7fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:58.129605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-13498886-defd-4cc2-a9fb-fc4fab8e7fd4 None None] MFA Rules not processed for user `1d96bcf28939461487e0fd8a9e516811`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:58.130898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-13498886-defd-4cc2-a9fb-fc4fab8e7fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:58.185507 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:58.232327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 650/1318] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:58.525319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec18895-6916-4cae-9508-e3e4d4b7e63f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:58.526921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-aec18895-6916-4cae-9508-e3e4d4b7e63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:58.634917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec18895-6916-4cae-9508-e3e4d4b7e63f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:58.637868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:58.713294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f963cd5-f5f1-41c3-97d3-d63692091eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:58.714023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963cd5-f5f1-41c3-97d3-d63692091eca service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:58.716717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963cd5-f5f1-41c3-97d3-d63692091eca service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:58.716717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963cd5-f5f1-41c3-97d3-d63692091eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:58.716717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5f963cd5-f5f1-41c3-97d3-d63692091eca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:58.834562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f963cd5-f5f1-41c3-97d3-d63692091eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:58.835920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f963cd5-f5f1-41c3-97d3-d63692091eca service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:58.855744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 669/1319] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:01.897936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc45f-4a95-4770-a7c3-6cb08621e3e7 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.898529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc45f-4a95-4770-a7c3-6cb08621e3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.898529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc45f-4a95-4770-a7c3-6cb08621e3e7 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.899849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 651/1320] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:01.906111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d5c4f-8a89-43ac-980a-e5776970cbb7 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.906592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d5c4f-8a89-43ac-980a-e5776970cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.906778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d5c4f-8a89-43ac-980a-e5776970cbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.952724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:01.979126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a36d5c4f-8a89-43ac-980a-e5776970cbb7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:01.980362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a36d5c4f-8a89-43ac-980a-e5776970cbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:02.079716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 670/1321] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:02.362886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc45f-4a95-4770-a7c3-6cb08621e3e7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:02.364432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b12dc45f-4a95-4770-a7c3-6cb08621e3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:02.470955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc45f-4a95-4770-a7c3-6cb08621e3e7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:02.548261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff885ff5-8ccd-42d2-b213-04024ea69217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:02.548878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff885ff5-8ccd-42d2-b213-04024ea69217 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:02.549161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff885ff5-8ccd-42d2-b213-04024ea69217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:02.549434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff885ff5-8ccd-42d2-b213-04024ea69217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:02.551260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ff885ff5-8ccd-42d2-b213-04024ea69217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:02.559505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:02.666990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff885ff5-8ccd-42d2-b213-04024ea69217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:02.668354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff885ff5-8ccd-42d2-b213-04024ea69217 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:02.688240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 652/1322] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:05.839210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3ca0-24e8-4998-91a9-21b303cb3a50 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.839880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3ca0-24e8-4998-91a9-21b303cb3a50 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.839880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3ca0-24e8-4998-91a9-21b303cb3a50 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.841113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 671/1323] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:05.847479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741bcb-b74c-4395-b82a-98c8607ac8e4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.847990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741bcb-b74c-4395-b82a-98c8607ac8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.848133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741bcb-b74c-4395-b82a-98c8607ac8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.924775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7c741bcb-b74c-4395-b82a-98c8607ac8e4 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:05.926026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7c741bcb-b74c-4395-b82a-98c8607ac8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.988867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:06.032468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 653/1324] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:06.324747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3ca0-24e8-4998-91a9-21b303cb3a50 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.326334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-aebd3ca0-24e8-4998-91a9-21b303cb3a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.450573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3ca0-24e8-4998-91a9-21b303cb3a50 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:06.464286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:06.527893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06709199-6d4d-4d01-9bab-095ecfe4487e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.528506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06709199-6d4d-4d01-9bab-095ecfe4487e service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.528790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06709199-6d4d-4d01-9bab-095ecfe4487e service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.529064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06709199-6d4d-4d01-9bab-095ecfe4487e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.530782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-06709199-6d4d-4d01-9bab-095ecfe4487e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.646747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06709199-6d4d-4d01-9bab-095ecfe4487e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.648194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06709199-6d4d-4d01-9bab-095ecfe4487e service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.666666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 672/1325] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:08.689176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5c84d-149b-4f21-bb96-1855dcc63819 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.689686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5c84d-149b-4f21-bb96-1855dcc63819 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.689686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5c84d-149b-4f21-bb96-1855dcc63819 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.691044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 654/1326] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:41:08.696912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4455c53-6812-4154-beb4-a84af13d85d7 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.697366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4455c53-6812-4154-beb4-a84af13d85d7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.697578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4455c53-6812-4154-beb4-a84af13d85d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.752216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:08.771618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f4455c53-6812-4154-beb4-a84af13d85d7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:08.772860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f4455c53-6812-4154-beb4-a84af13d85d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.879011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 673/1327] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:09.167936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5c84d-149b-4f21-bb96-1855dcc63819 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.169532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-02c5c84d-149b-4f21-bb96-1855dcc63819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.290050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5c84d-149b-4f21-bb96-1855dcc63819 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.366361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69002fad-2ff2-4080-8b73-73e1c964c029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.366963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002fad-2ff2-4080-8b73-73e1c964c029 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.367236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002fad-2ff2-4080-8b73-73e1c964c029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.367504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002fad-2ff2-4080-8b73-73e1c964c029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.369112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-69002fad-2ff2-4080-8b73-73e1c964c029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.387276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:09.485710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69002fad-2ff2-4080-8b73-73e1c964c029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.487088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69002fad-2ff2-4080-8b73-73e1c964c029 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.506724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 655/1328] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:11.508408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dabb4b-560f-4a88-a4ea-d2cfcfdd94ea None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.509028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dabb4b-560f-4a88-a4ea-d2cfcfdd94ea None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.509028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dabb4b-560f-4a88-a4ea-d2cfcfdd94ea None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.510694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 674/1329] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:11.517505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e02b3-f8d5-432f-a447-8f6052b51b6d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.518099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e02b3-f8d5-432f-a447-8f6052b51b6d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.518368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e02b3-f8d5-432f-a447-8f6052b51b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.602729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-116e02b3-f8d5-432f-a447-8f6052b51b6d None None] MFA Rules not processed for user `1d96bcf28939461487e0fd8a9e516811`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:11.603945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-116e02b3-f8d5-432f-a447-8f6052b51b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:11.674647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:11.711441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 656/1330] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:12.000020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dabb4b-560f-4a88-a4ea-d2cfcfdd94ea None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:12.001743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-30dabb4b-560f-4a88-a4ea-d2cfcfdd94ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:12.117001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dabb4b-560f-4a88-a4ea-d2cfcfdd94ea None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:12.138878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:12.198183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f679b-8278-4296-a298-69722898b31c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:12.198708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f679b-8278-4296-a298-69722898b31c service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:12.198986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f679b-8278-4296-a298-69722898b31c service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:12.199254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f679b-8278-4296-a298-69722898b31c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:12.200905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-863f679b-8278-4296-a298-69722898b31c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:12.318735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f679b-8278-4296-a298-69722898b31c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:12.320171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f679b-8278-4296-a298-69722898b31c service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:12.339453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 675/1331] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:14.773481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0bd72-8a86-424b-b776-a7c3f6b29573 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.774032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0bd72-8a86-424b-b776-a7c3f6b29573 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.774162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0bd72-8a86-424b-b776-a7c3f6b29573 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.775488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 657/1332] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:14.782164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1be322-6cdd-4066-aa6d-81453a28cfbe None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.782674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1be322-6cdd-4066-aa6d-81453a28cfbe None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.782774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1be322-6cdd-4066-aa6d-81453a28cfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.849460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:14.852281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-0a1be322-6cdd-4066-aa6d-81453a28cfbe None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:14.853525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0a1be322-6cdd-4066-aa6d-81453a28cfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:14.955752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 676/1333] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:15.237419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0bd72-8a86-424b-b776-a7c3f6b29573 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.239062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8bb0bd72-8a86-424b-b776-a7c3f6b29573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:15.346047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0bd72-8a86-424b-b776-a7c3f6b29573 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.423515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be6349-90ea-46a6-83df-3bf58a9f0820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.424111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be6349-90ea-46a6-83df-3bf58a9f0820 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.424387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be6349-90ea-46a6-83df-3bf58a9f0820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.424652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be6349-90ea-46a6-83df-3bf58a9f0820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.426378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d2be6349-90ea-46a6-83df-3bf58a9f0820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:15.453620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:15.538891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be6349-90ea-46a6-83df-3bf58a9f0820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.540273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be6349-90ea-46a6-83df-3bf58a9f0820 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.560284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 658/1334] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:17.538843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df071e7b-b667-4a4f-b99c-f0f04e6e567c None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.539463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df071e7b-b667-4a4f-b99c-f0f04e6e567c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.539463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df071e7b-b667-4a4f-b99c-f0f04e6e567c None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.540742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 677/1335] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:17.547409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad9715-1229-40c7-af52-9b10a26aade1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.547930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad9715-1229-40c7-af52-9b10a26aade1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.547930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad9715-1229-40c7-af52-9b10a26aade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.616816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a7ad9715-1229-40c7-af52-9b10a26aade1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:17.618138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a7ad9715-1229-40c7-af52-9b10a26aade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:17.696467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:17.722920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 659/1336] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:18.006110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df071e7b-b667-4a4f-b99c-f0f04e6e567c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:18.007778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-df071e7b-b667-4a4f-b99c-f0f04e6e567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:18.112277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df071e7b-b667-4a4f-b99c-f0f04e6e567c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:18.141496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:18.187125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cd0d7-c125-4f32-ae26-0760223432fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:18.187648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd0d7-c125-4f32-ae26-0760223432fd service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:18.187921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd0d7-c125-4f32-ae26-0760223432fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:18.188192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd0d7-c125-4f32-ae26-0760223432fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:18.189804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c35cd0d7-c125-4f32-ae26-0760223432fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:18.298984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cd0d7-c125-4f32-ae26-0760223432fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:18.300804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cd0d7-c125-4f32-ae26-0760223432fd service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:18.319144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 678/1337] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:20.817599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531193-9f57-4697-bec3-c1e3348f6741 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.818164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531193-9f57-4697-bec3-c1e3348f6741 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.818287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531193-9f57-4697-bec3-c1e3348f6741 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.819645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 660/1338] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:20.825261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580d26-f081-4f53-8589-db3da58f3f1a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.825668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580d26-f081-4f53-8589-db3da58f3f1a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.825818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580d26-f081-4f53-8589-db3da58f3f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.892162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-22580d26-f081-4f53-8589-db3da58f3f1a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:20.893375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-22580d26-f081-4f53-8589-db3da58f3f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:20.902211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:20.996770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 679/1339] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:21.283036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70531193-9f57-4697-bec3-c1e3348f6741 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:21.284688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-70531193-9f57-4697-bec3-c1e3348f6741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:21.396187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70531193-9f57-4697-bec3-c1e3348f6741 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:21.469261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68846ba-be75-4869-8531-f0515f32e1d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:21.469889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68846ba-be75-4869-8531-f0515f32e1d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:21.470173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68846ba-be75-4869-8531-f0515f32e1d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:21.470445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68846ba-be75-4869-8531-f0515f32e1d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:21.472062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a68846ba-be75-4869-8531-f0515f32e1d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:21.507485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:21.582934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68846ba-be75-4869-8531-f0515f32e1d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:21.584290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68846ba-be75-4869-8531-f0515f32e1d8 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:21.602943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 661/1340] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:25.704329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a56a1-c241-4366-8df8-b21c88fe5580 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:25.704869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a56a1-c241-4366-8df8-b21c88fe5580 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:25.704869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a56a1-c241-4366-8df8-b21c88fe5580 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:25.706177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 680/1341] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:25.711855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd4bd-f7aa-419e-a718-974ef038f173 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:25.717166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd4bd-f7aa-419e-a718-974ef038f173 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:25.717166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd4bd-f7aa-419e-a718-974ef038f173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:25.782656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-31bcd4bd-f7aa-419e-a718-974ef038f173 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:25.783951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-31bcd4bd-f7aa-419e-a718-974ef038f173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:25.875645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:25.887879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 662/1342] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:26.180268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a56a1-c241-4366-8df8-b21c88fe5580 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.181838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d87a56a1-c241-4366-8df8-b21c88fe5580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:26.292514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a56a1-c241-4366-8df8-b21c88fe5580 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.333097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:26.370646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8c3a9-9981-40df-ac7e-771e4c6babcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.371263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8c3a9-9981-40df-ac7e-771e4c6babcc service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.371546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8c3a9-9981-40df-ac7e-771e4c6babcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.371822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8c3a9-9981-40df-ac7e-771e4c6babcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.374320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-12b8c3a9-9981-40df-ac7e-771e4c6babcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:26.488914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8c3a9-9981-40df-ac7e-771e4c6babcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.490402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8c3a9-9981-40df-ac7e-771e4c6babcc service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.510430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 681/1343] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:29.459889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f47073-695b-476c-8e14-b8f12c3e2a40 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.460456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f47073-695b-476c-8e14-b8f12c3e2a40 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.460456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f47073-695b-476c-8e14-b8f12c3e2a40 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.461795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 663/1344] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:29.467673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb30b-1baf-48cc-9b61-1dae8dc6c317 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.468182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb30b-1baf-48cc-9b61-1dae8dc6c317 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.468285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb30b-1baf-48cc-9b61-1dae8dc6c317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.537828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a07cb30b-1baf-48cc-9b61-1dae8dc6c317 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:29.538992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a07cb30b-1baf-48cc-9b61-1dae8dc6c317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.557843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:29.652322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 682/1345] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:29.943552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f47073-695b-476c-8e14-b8f12c3e2a40 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:29.945106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f4f47073-695b-476c-8e14-b8f12c3e2a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.056893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f47073-695b-476c-8e14-b8f12c3e2a40 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:30.135333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf21a77-9c4d-467c-a599-fe5b35666055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:30.136090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf21a77-9c4d-467c-a599-fe5b35666055 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.136512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf21a77-9c4d-467c-a599-fe5b35666055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.136812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf21a77-9c4d-467c-a599-fe5b35666055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.138493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4bf21a77-9c4d-467c-a599-fe5b35666055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.184946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:30.258025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf21a77-9c4d-467c-a599-fe5b35666055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:30.259513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf21a77-9c4d-467c-a599-fe5b35666055 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:30.278114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 664/1346] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:34.499385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac1019-3414-468a-a1d9-0110b01b6cba None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.499871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac1019-3414-468a-a1d9-0110b01b6cba None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.499995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac1019-3414-468a-a1d9-0110b01b6cba None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.501293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 683/1347] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:34.506692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218469b6-39a7-4727-b66f-4f895d0fe0a4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.507107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218469b6-39a7-4727-b66f-4f895d0fe0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.507251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218469b6-39a7-4727-b66f-4f895d0fe0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.584554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-218469b6-39a7-4727-b66f-4f895d0fe0a4 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:34.585813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-218469b6-39a7-4727-b66f-4f895d0fe0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:34.701521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:34.704173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 665/1348] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:34.989976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac1019-3414-468a-a1d9-0110b01b6cba None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.991580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5bac1019-3414-468a-a1d9-0110b01b6cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:35.114946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac1019-3414-468a-a1d9-0110b01b6cba None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:35.165699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:35.193741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecb56e-42ea-4b7f-bcd7-8d95c47959a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:35.194411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecb56e-42ea-4b7f-bcd7-8d95c47959a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:35.194686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecb56e-42ea-4b7f-bcd7-8d95c47959a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:35.194955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecb56e-42ea-4b7f-bcd7-8d95c47959a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:35.196596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-adecb56e-42ea-4b7f-bcd7-8d95c47959a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:35.307451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecb56e-42ea-4b7f-bcd7-8d95c47959a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:35.308876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecb56e-42ea-4b7f-bcd7-8d95c47959a6 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:35.327187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 684/1349] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:40.285789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1386f1-067d-48c1-92c0-29524d949b64 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.286319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1386f1-067d-48c1-92c0-29524d949b64 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.286427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1386f1-067d-48c1-92c0-29524d949b64 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.288490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 666/1350] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:40.293591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d487e-e13c-4722-948d-f5569a03779a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.294136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d487e-e13c-4722-948d-f5569a03779a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.294425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d487e-e13c-4722-948d-f5569a03779a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.398341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-545d487e-e13c-4722-948d-f5569a03779a None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:40.399682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-545d487e-e13c-4722-948d-f5569a03779a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:40.434062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:40.599298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 685/1351] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:45.129690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b821d4-d37f-4be3-8201-58c49cf94b3c None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:45.130479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b821d4-d37f-4be3-8201-58c49cf94b3c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:45.130479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b821d4-d37f-4be3-8201-58c49cf94b3c None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:45.131527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 667/1352] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:45.138621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3dff2-3bf0-4ca4-b5a8-3607a98bca73 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:45.139167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3dff2-3bf0-4ca4-b5a8-3607a98bca73 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:45.139401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3dff2-3bf0-4ca4-b5a8-3607a98bca73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:45.236947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-5da3dff2-3bf0-4ca4-b5a8-3607a98bca73 None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:45.238502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5da3dff2-3bf0-4ca4-b5a8-3607a98bca73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:45.330065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:45.373397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 686/1353] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:41:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:46.535693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa866e80-6722-4833-b4ef-2ab15b23ddc2 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:46.536553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa866e80-6722-4833-b4ef-2ab15b23ddc2 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:46.536812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa866e80-6722-4833-b4ef-2ab15b23ddc2 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:46.538207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 668/1354] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:46.543630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a687d-a529-49be-aa11-a90c18e8333f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:46.544178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a687d-a529-49be-aa11-a90c18e8333f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:46.544178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a687d-a529-49be-aa11-a90c18e8333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:46.623888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b04a687d-a529-49be-aa11-a90c18e8333f None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:46.625100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b04a687d-a529-49be-aa11-a90c18e8333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:46.738725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 687/1355] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:41:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:50.336400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e723f-af77-406d-b266-e87d136023d8 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:50.336980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e723f-af77-406d-b266-e87d136023d8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:50.336980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e723f-af77-406d-b266-e87d136023d8 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:50.338352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 669/1356] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:50.344979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30c18a-7e3f-4954-9f0d-2ad08c1203e3 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:50.345450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30c18a-7e3f-4954-9f0d-2ad08c1203e3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:50.345581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30c18a-7e3f-4954-9f0d-2ad08c1203e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:50.393240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:50.433965 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7c30c18a-7e3f-4954-9f0d-2ad08c1203e3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:50.435704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7c30c18a-7e3f-4954-9f0d-2ad08c1203e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:50.581053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 688/1357] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:50.886571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d935a97-f3c2-41aa-93c6-409172afd21f None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:50.887374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d935a97-f3c2-41aa-93c6-409172afd21f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:50.887959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d935a97-f3c2-41aa-93c6-409172afd21f None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:50.890857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 670/1358] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 24 17:41:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:50.897806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8e65-6c8e-4f77-9045-9624cbe6aa93 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:50.898419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8e65-6c8e-4f77-9045-9624cbe6aa93 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:50.898851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8e65-6c8e-4f77-9045-9624cbe6aa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:51.004760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-221c8e65-6c8e-4f77-9045-9624cbe6aa93 None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:51.006147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-221c8e65-6c8e-4f77-9045-9624cbe6aa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:51.067514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:51.148129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 689/1359] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 24 17:41:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:51.149422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d935a97-f3c2-41aa-93c6-409172afd21f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:51.151306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4d935a97-f3c2-41aa-93c6-409172afd21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:51.322064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d935a97-f3c2-41aa-93c6-409172afd21f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:51.399260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:51.442207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbcd85-d098-4f43-8d53-1bbe5e609083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:51.442816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbcd85-d098-4f43-8d53-1bbe5e609083 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:51.443095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbcd85-d098-4f43-8d53-1bbe5e609083 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:51.443365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbcd85-d098-4f43-8d53-1bbe5e609083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:51.444990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-fadbcd85-d098-4f43-8d53-1bbe5e609083 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:51.540373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbcd85-d098-4f43-8d53-1bbe5e609083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:51.542433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbcd85-d098-4f43-8d53-1bbe5e609083 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:51.563389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 671/1360] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:53.739036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36894a9-d13c-4301-b957-d93ee4098438 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:53.739733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36894a9-d13c-4301-b957-d93ee4098438 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:53.739733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36894a9-d13c-4301-b957-d93ee4098438 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:53.740973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 690/1361] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:53.747426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31e5a-b42b-49e1-87ca-dce2401b2034 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:53.748103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31e5a-b42b-49e1-87ca-dce2401b2034 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:53.748103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31e5a-b42b-49e1-87ca-dce2401b2034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:53.821246 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2dd31e5a-b42b-49e1-87ca-dce2401b2034 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:53.822496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2dd31e5a-b42b-49e1-87ca-dce2401b2034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:53.864303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:53.942574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 672/1362] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:54.236033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36894a9-d13c-4301-b957-d93ee4098438 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:54.237585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b36894a9-d13c-4301-b957-d93ee4098438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:54.399595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36894a9-d13c-4301-b957-d93ee4098438 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:54.410648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:54.521257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ba51f-ae48-42f7-9326-9b730312e326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:54.521780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba51f-ae48-42f7-9326-9b730312e326 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:54.522085 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba51f-ae48-42f7-9326-9b730312e326 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:54.522360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba51f-ae48-42f7-9326-9b730312e326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:54.523980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7f2ba51f-ae48-42f7-9326-9b730312e326 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:54.630611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ba51f-ae48-42f7-9326-9b730312e326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:54.632225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ba51f-ae48-42f7-9326-9b730312e326 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:54.651462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 691/1363] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:57.011388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f989-fa69-43c4-bc1b-fe45ec59e46c None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:57.012127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f989-fa69-43c4-bc1b-fe45ec59e46c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:57.012127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f989-fa69-43c4-bc1b-fe45ec59e46c None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:57.013424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 673/1364] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:57.019132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62525-0863-4109-be2d-daf1c5d4d389 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:57.019590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62525-0863-4109-be2d-daf1c5d4d389 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:57.019816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62525-0863-4109-be2d-daf1c5d4d389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:57.037650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:57.096980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-75c62525-0863-4109-be2d-daf1c5d4d389 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:57.098246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-75c62525-0863-4109-be2d-daf1c5d4d389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:57.212893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 692/1365] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:57.510477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c42675-8dcf-454e-ba0f-61dfd68d11b7 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:57.511069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c42675-8dcf-454e-ba0f-61dfd68d11b7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:57.511438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c42675-8dcf-454e-ba0f-61dfd68d11b7 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:57.513674 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 674/1366] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 24 17:41:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:57.519410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324787-1de9-4a0b-8490-83f198e0ec5c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:57.519705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324787-1de9-4a0b-8490-83f198e0ec5c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:57.521758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324787-1de9-4a0b-8490-83f198e0ec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:57.607499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-69324787-1de9-4a0b-8490-83f198e0ec5c None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:57.608721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-69324787-1de9-4a0b-8490-83f198e0ec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:57.646299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:57.741269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 693/1367] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 24 17:41:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:57.750195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c42675-8dcf-454e-ba0f-61dfd68d11b7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:57.751913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-65c42675-8dcf-454e-ba0f-61dfd68d11b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:57.922818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c42675-8dcf-454e-ba0f-61dfd68d11b7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:57.945357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:58.050472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e6947-2279-4f61-96e3-4365fd182c04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:58.051164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6947-2279-4f61-96e3-4365fd182c04 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:58.051254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6947-2279-4f61-96e3-4365fd182c04 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:58.051504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6947-2279-4f61-96e3-4365fd182c04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:58.053309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ad8e6947-2279-4f61-96e3-4365fd182c04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:58.158629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e6947-2279-4f61-96e3-4365fd182c04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:58.160601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e6947-2279-4f61-96e3-4365fd182c04 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:58.181692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 675/1368] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:58.651650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe0e29-8f2a-436b-b9e3-d335acd1c02d None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:58.652184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe0e29-8f2a-436b-b9e3-d335acd1c02d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:58.652184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe0e29-8f2a-436b-b9e3-d335acd1c02d None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:58.654230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 694/1369] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:58.659444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b42dd-26b4-4d25-b8a5-67ce49663258 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:58.659738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b42dd-26b4-4d25-b8a5-67ce49663258 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:58.660017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b42dd-26b4-4d25-b8a5-67ce49663258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:58.696088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:58.736903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ba5b42dd-26b4-4d25-b8a5-67ce49663258 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:58.738529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ba5b42dd-26b4-4d25-b8a5-67ce49663258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:58.851254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 676/1370] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:59.400034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013afa70-587d-4bd4-ab73-43bc0a3925c5 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:59.400320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013afa70-587d-4bd4-ab73-43bc0a3925c5 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:59.400595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013afa70-587d-4bd4-ab73-43bc0a3925c5 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:59.402130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 695/1371] 10.222.0.76 () {54 vars in 964 bytes} [Wed Aug 24 17:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:59.407315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1195b-7c9d-4927-9441-f988f93861c5 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:59.407832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1195b-7c9d-4927-9441-f988f93861c5 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:59.407925 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1195b-7c9d-4927-9441-f988f93861c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:59.488099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2bc1195b-7c9d-4927-9441-f988f93861c5 None None] MFA Rules not processed for user `6c7a064cf91243648c6cc6d6ca576505`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:59.489308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2bc1195b-7c9d-4927-9441-f988f93861c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:59.542688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:59.578681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 677/1372] 10.222.0.76 () {58 vars in 1061 bytes} [Wed Aug 24 17:41:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:59.587247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013afa70-587d-4bd4-ab73-43bc0a3925c5 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:59.588870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-013afa70-587d-4bd4-ab73-43bc0a3925c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:59.703490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013afa70-587d-4bd4-ab73-43bc0a3925c5 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:59.746202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:59.782926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89815196-ca34-41e8-8123-06ecff2f25b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:59.783684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89815196-ca34-41e8-8123-06ecff2f25b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:59.784064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89815196-ca34-41e8-8123-06ecff2f25b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:59.784448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89815196-ca34-41e8-8123-06ecff2f25b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:59.786187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-89815196-ca34-41e8-8123-06ecff2f25b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:59.903699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89815196-ca34-41e8-8123-06ecff2f25b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:59.905187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89815196-ca34-41e8-8123-06ecff2f25b5 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:59.925779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 696/1373] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:00.377932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416eac8-5be9-4b04-8b6c-0f68fc005a9f None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:00.378222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416eac8-5be9-4b04-8b6c-0f68fc005a9f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:00.378676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416eac8-5be9-4b04-8b6c-0f68fc005a9f None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:00.380047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 678/1374] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:00.385541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752277-4b2d-453e-b45d-385318dc1406 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:00.385832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752277-4b2d-453e-b45d-385318dc1406 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:00.386134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752277-4b2d-453e-b45d-385318dc1406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:00.467274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-05752277-4b2d-453e-b45d-385318dc1406 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:00.469321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-05752277-4b2d-453e-b45d-385318dc1406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:00.493119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:00.602294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 697/1375] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:00.883591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416eac8-5be9-4b04-8b6c-0f68fc005a9f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:00.885156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7416eac8-5be9-4b04-8b6c-0f68fc005a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.053643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416eac8-5be9-4b04-8b6c-0f68fc005a9f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:01.109676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:01.178332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6765ee0-e1de-41c8-b8d0-1255fcf66e47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:01.178952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6765ee0-e1de-41c8-b8d0-1255fcf66e47 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.179228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6765ee0-e1de-41c8-b8d0-1255fcf66e47 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.179523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6765ee0-e1de-41c8-b8d0-1255fcf66e47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.181292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a6765ee0-e1de-41c8-b8d0-1255fcf66e47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.276596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6765ee0-e1de-41c8-b8d0-1255fcf66e47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:01.279215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6765ee0-e1de-41c8-b8d0-1255fcf66e47 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:01.299222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 679/1376] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:03.389040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718765-b126-496b-b098-479e2f1cf83f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.389536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718765-b126-496b-b098-479e2f1cf83f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.389536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718765-b126-496b-b098-479e2f1cf83f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.390872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 698/1377] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:03.396113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5135d-2e8b-4bcf-a644-22bcc7ec5daa None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.396541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5135d-2e8b-4bcf-a644-22bcc7ec5daa None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.396661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5135d-2e8b-4bcf-a644-22bcc7ec5daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.464183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-23d5135d-2e8b-4bcf-a644-22bcc7ec5daa None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:03.465783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-23d5135d-2e8b-4bcf-a644-22bcc7ec5daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.473153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:03.572457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 680/1378] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:03.879089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8718765-b126-496b-b098-479e2f1cf83f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:03.880722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e8718765-b126-496b-b098-479e2f1cf83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:04.050273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8718765-b126-496b-b098-479e2f1cf83f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.054297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:04.180931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3950ed-2d6f-424d-9e5a-221b6c7ab58e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.181805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3950ed-2d6f-424d-9e5a-221b6c7ab58e service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.182291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3950ed-2d6f-424d-9e5a-221b6c7ab58e service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.182727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3950ed-2d6f-424d-9e5a-221b6c7ab58e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.185400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ba3950ed-2d6f-424d-9e5a-221b6c7ab58e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:04.298876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3950ed-2d6f-424d-9e5a-221b6c7ab58e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.300750 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3950ed-2d6f-424d-9e5a-221b6c7ab58e service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.320346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 699/1379] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:06.408375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0bb7-104d-42a5-8e7b-1fb745cd40b2 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.408942 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0bb7-104d-42a5-8e7b-1fb745cd40b2 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.408942 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0bb7-104d-42a5-8e7b-1fb745cd40b2 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.411244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 681/1380] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:06.415739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1c7a-0ca4-4ccd-828f-24a816ea77a9 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.416165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1c7a-0ca4-4ccd-828f-24a816ea77a9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.416357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1c7a-0ca4-4ccd-828f-24a816ea77a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.428300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:06.488654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f12e1c7a-0ca4-4ccd-828f-24a816ea77a9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:06.489939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f12e1c7a-0ca4-4ccd-828f-24a816ea77a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:06.611805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 700/1381] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:06.911307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad0bb7-104d-42a5-8e7b-1fb745cd40b2 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.914937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2dad0bb7-104d-42a5-8e7b-1fb745cd40b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:07.068831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:07.079532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad0bb7-104d-42a5-8e7b-1fb745cd40b2 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:07.202655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abd301-3118-4344-a0a6-5d77773a193e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:07.203274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abd301-3118-4344-a0a6-5d77773a193e service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:07.203664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abd301-3118-4344-a0a6-5d77773a193e service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:07.203943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abd301-3118-4344-a0a6-5d77773a193e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:07.205568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-69abd301-3118-4344-a0a6-5d77773a193e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:07.301151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abd301-3118-4344-a0a6-5d77773a193e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:07.302926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abd301-3118-4344-a0a6-5d77773a193e service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:07.314500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:07.328060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 682/1382] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:09.472894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a3c3-8f40-44c7-908f-174d3ed5705b None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:09.473456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a3c3-8f40-44c7-908f-174d3ed5705b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:09.473456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a3c3-8f40-44c7-908f-174d3ed5705b None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:09.474852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 701/1383] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:09.480331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bfa8-1d3c-48bb-966b-4be1edc75ad4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:09.480981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bfa8-1d3c-48bb-966b-4be1edc75ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:09.480981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bfa8-1d3c-48bb-966b-4be1edc75ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:09.550648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-8fb8bfa8-1d3c-48bb-966b-4be1edc75ad4 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:09.551871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8fb8bfa8-1d3c-48bb-966b-4be1edc75ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:09.655073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 683/1384] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:10.018698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a3c3-8f40-44c7-908f-174d3ed5705b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:10.020698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4ee6a3c3-8f40-44c7-908f-174d3ed5705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:10.121894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:10.188202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a3c3-8f40-44c7-908f-174d3ed5705b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:10.309580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805858f-10c8-4080-b265-e9db3118bcb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:10.310208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805858f-10c8-4080-b265-e9db3118bcb3 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:10.310573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805858f-10c8-4080-b265-e9db3118bcb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:10.310752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805858f-10c8-4080-b265-e9db3118bcb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:10.312451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9805858f-10c8-4080-b265-e9db3118bcb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:10.365071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:10.408353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805858f-10c8-4080-b265-e9db3118bcb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:10.410029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805858f-10c8-4080-b265-e9db3118bcb3 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:10.429301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 702/1385] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:12.385116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04ba5-830b-462b-ba51-fc8543700a7e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.385657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04ba5-830b-462b-ba51-fc8543700a7e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.385657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04ba5-830b-462b-ba51-fc8543700a7e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.387496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 684/1386] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:12.393254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5782482-a141-4af2-b21e-3d5f37d619c8 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.393736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5782482-a141-4af2-b21e-3d5f37d619c8 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.394076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5782482-a141-4af2-b21e-3d5f37d619c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.479881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d5782482-a141-4af2-b21e-3d5f37d619c8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:12.481700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d5782482-a141-4af2-b21e-3d5f37d619c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:12.585227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 703/1387] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:12.955994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a04ba5-830b-462b-ba51-fc8543700a7e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:12.957619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-10a04ba5-830b-462b-ba51-fc8543700a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:13.051258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:13.140924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a04ba5-830b-462b-ba51-fc8543700a7e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.289077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94091b3-7a26-4246-b24c-c570a7df57c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.289868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94091b3-7a26-4246-b24c-c570a7df57c7 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.290305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94091b3-7a26-4246-b24c-c570a7df57c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.290893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94091b3-7a26-4246-b24c-c570a7df57c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.293516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b94091b3-7a26-4246-b24c-c570a7df57c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:13.345894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:13.411605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94091b3-7a26-4246-b24c-c570a7df57c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.413990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94091b3-7a26-4246-b24c-c570a7df57c7 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.439711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 685/1388] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:15.519262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef0a5-68f7-4241-a947-742289b0f691 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.519852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef0a5-68f7-4241-a947-742289b0f691 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.519852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef0a5-68f7-4241-a947-742289b0f691 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.521158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 704/1389] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:15.526473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade77e1-272d-4302-9252-b217be5aac12 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.526953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade77e1-272d-4302-9252-b217be5aac12 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.527060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade77e1-272d-4302-9252-b217be5aac12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.597614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9ade77e1-272d-4302-9252-b217be5aac12 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:15.598870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9ade77e1-272d-4302-9252-b217be5aac12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:15.602672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7935a6a-91e8-4e62-9d2f-887a5d9c3fe3 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.602969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7935a6a-91e8-4e62-9d2f-887a5d9c3fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.603257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7935a6a-91e8-4e62-9d2f-887a5d9c3fe3 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.604865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 705/1390] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:42:15.610213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cde05-deda-4ffd-b005-b70865b5c515 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.610502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cde05-deda-4ffd-b005-b70865b5c515 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.610775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cde05-deda-4ffd-b005-b70865b5c515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.646791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:15.679395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-207cde05-deda-4ffd-b005-b70865b5c515 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:15.680612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-207cde05-deda-4ffd-b005-b70865b5c515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:15.703332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 686/1391] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:15.785096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 706/1392] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:15.991237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade77e1-272d-4302-9252-b217be5aac12 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:15.992469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9ade77e1-272d-4302-9252-b217be5aac12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:16.006840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:16.088622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb13d70-4ba8-458a-9a09-92bd2e297539 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.089019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb13d70-4ba8-458a-9a09-92bd2e297539 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.089829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb13d70-4ba8-458a-9a09-92bd2e297539 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.091381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 707/1393] 10.222.0.76 () {54 vars in 967 bytes} [Wed Aug 24 17:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:42:16.096595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a782d-76f1-4257-83b5-a8cefcbbcea8 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.096866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a782d-76f1-4257-83b5-a8cefcbbcea8 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.097143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a782d-76f1-4257-83b5-a8cefcbbcea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.116736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade77e1-272d-4302-9252-b217be5aac12 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.186983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c77a782d-76f1-4257-83b5-a8cefcbbcea8 None None] MFA Rules not processed for user `b4d874536170494496769b4f37234b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:16.188207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c77a782d-76f1-4257-83b5-a8cefcbbcea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:16.239938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511f71c-3d25-4da6-9d1e-e4f6f3289137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.240332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:16.241121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f71c-3d25-4da6-9d1e-e4f6f3289137 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.241420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f71c-3d25-4da6-9d1e-e4f6f3289137 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.241760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f71c-3d25-4da6-9d1e-e4f6f3289137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.243991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a511f71c-3d25-4da6-9d1e-e4f6f3289137 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:16.253678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:16.269410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 708/1394] 10.222.0.76 () {58 vars in 1064 bytes} [Wed Aug 24 17:42:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:16.278005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a782d-76f1-4257-83b5-a8cefcbbcea8 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:16.285595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a782d-76f1-4257-83b5-a8cefcbbcea8 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.337411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511f71c-3d25-4da6-9d1e-e4f6f3289137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.339593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511f71c-3d25-4da6-9d1e-e4f6f3289137 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.360795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 687/1395] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:16.363853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f53439-88e0-42bd-84f6-48bd3fd1f97b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d874536170494496769b4f37234b2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.364456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f53439-88e0-42bd-84f6-48bd3fd1f97b service glance] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.364732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f53439-88e0-42bd-84f6-48bd3fd1f97b service glance] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.365001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f53439-88e0-42bd-84f6-48bd3fd1f97b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.367141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c3f53439-88e0-42bd-84f6-48bd3fd1f97b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:16.481430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f53439-88e0-42bd-84f6-48bd3fd1f97b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.482927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f53439-88e0-42bd-84f6-48bd3fd1f97b service glance] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.497084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:16.501742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 709/1396] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:17.427249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65cfab-fec5-4592-90b6-e0ee20539e0d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:17.427733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65cfab-fec5-4592-90b6-e0ee20539e0d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:17.427733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65cfab-fec5-4592-90b6-e0ee20539e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:17.498359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7f65cfab-fec5-4592-90b6-e0ee20539e0d None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:17.499601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7f65cfab-fec5-4592-90b6-e0ee20539e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:17.585590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:17.589637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 688/1397] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:42:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:17.641570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2667f3c-5250-4717-b97c-9767c1535bcd None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:17.642143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2667f3c-5250-4717-b97c-9767c1535bcd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:17.642143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2667f3c-5250-4717-b97c-9767c1535bcd None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:17.643527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 710/1398] 10.222.0.76 () {54 vars in 943 bytes} [Wed Aug 24 17:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:17.650902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2c6f8-f9f9-4e2b-a11d-755e06bc2670 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:17.651222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2c6f8-f9f9-4e2b-a11d-755e06bc2670 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:17.652073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2c6f8-f9f9-4e2b-a11d-755e06bc2670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:17.736158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2cb2c6f8-f9f9-4e2b-a11d-755e06bc2670 None None] MFA Rules not processed for user `e698e658de56495e8d501ca6dfd75623`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:17.737401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2cb2c6f8-f9f9-4e2b-a11d-755e06bc2670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:17.850921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 689/1399] 10.222.0.76 () {58 vars in 1040 bytes} [Wed Aug 24 17:42:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:17.861402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2667f3c-5250-4717-b97c-9767c1535bcd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:17.862960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e2667f3c-5250-4717-b97c-9767c1535bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:18.015351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2667f3c-5250-4717-b97c-9767c1535bcd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:18.104056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:18.124372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c195d1-4726-4f2e-bc8b-4c46aca46fab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:18.125048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c195d1-4726-4f2e-bc8b-4c46aca46fab service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:18.125372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c195d1-4726-4f2e-bc8b-4c46aca46fab service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:18.125650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c195d1-4726-4f2e-bc8b-4c46aca46fab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:18.127332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-19c195d1-4726-4f2e-bc8b-4c46aca46fab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:18.216295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c195d1-4726-4f2e-bc8b-4c46aca46fab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:18.217994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c195d1-4726-4f2e-bc8b-4c46aca46fab service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:18.226320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 711/1400] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:42:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:42:18.465590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8056a-253a-4a98-93ac-9ec19fb48735 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:18.466168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8056a-253a-4a98-93ac-9ec19fb48735 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:18.466168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8056a-253a-4a98-93ac-9ec19fb48735 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:18.467538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 690/1401] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:42:18.472998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a262e78-210c-49a2-9ecb-328b1a66b941 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:18.473293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a262e78-210c-49a2-9ecb-328b1a66b941 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:18.473575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a262e78-210c-49a2-9ecb-328b1a66b941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:18.545220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-6a262e78-210c-49a2-9ecb-328b1a66b941 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:18.546469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6a262e78-210c-49a2-9ecb-328b1a66b941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:18.622540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:18.653192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 712/1402] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:18.945608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8056a-253a-4a98-93ac-9ec19fb48735 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:18.947338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1db8056a-253a-4a98-93ac-9ec19fb48735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:19.013629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:19.101175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8056a-253a-4a98-93ac-9ec19fb48735 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:19.241381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aff34c-4b6c-4481-b0b7-6eb8fac07051 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:19.242144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff34c-4b6c-4481-b0b7-6eb8fac07051 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:19.242505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff34c-4b6c-4481-b0b7-6eb8fac07051 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:19.242930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff34c-4b6c-4481-b0b7-6eb8fac07051 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:19.246260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-30aff34c-4b6c-4481-b0b7-6eb8fac07051 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:19.361052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aff34c-4b6c-4481-b0b7-6eb8fac07051 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:19.362602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aff34c-4b6c-4481-b0b7-6eb8fac07051 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:19.381803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 691/1403] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:20.045341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:22.010613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59435c2-9edc-4daa-a9ee-25b23eef6585 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:22.011302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59435c2-9edc-4daa-a9ee-25b23eef6585 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:22.011302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59435c2-9edc-4daa-a9ee-25b23eef6585 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:22.016592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 713/1404] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:22.017969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6c1c-5b86-4885-afee-e72348f52534 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:22.018385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6c1c-5b86-4885-afee-e72348f52534 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:22.018584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6c1c-5b86-4885-afee-e72348f52534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:22.096681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-dbca6c1c-5b86-4885-afee-e72348f52534 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:22.097966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-dbca6c1c-5b86-4885-afee-e72348f52534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:22.202660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:22.209369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 692/1405] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:22.508014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59435c2-9edc-4daa-a9ee-25b23eef6585 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:22.509548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d59435c2-9edc-4daa-a9ee-25b23eef6585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:22.680386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59435c2-9edc-4daa-a9ee-25b23eef6585 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:22.747728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:22.818458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f96d9-bb3d-4e57-8320-5c5dfee509d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:22.819080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f96d9-bb3d-4e57-8320-5c5dfee509d6 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:22.819355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f96d9-bb3d-4e57-8320-5c5dfee509d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:22.819626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f96d9-bb3d-4e57-8320-5c5dfee509d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:22.821380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-603f96d9-bb3d-4e57-8320-5c5dfee509d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:22.934158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f96d9-bb3d-4e57-8320-5c5dfee509d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:22.935782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f96d9-bb3d-4e57-8320-5c5dfee509d6 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:22.955036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 714/1406] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:25.172978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9424e-90c7-4a3b-9c08-6703a55dcca3 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:25.173836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9424e-90c7-4a3b-9c08-6703a55dcca3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:25.174137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9424e-90c7-4a3b-9c08-6703a55dcca3 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:25.175652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 693/1407] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:25.181083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca737-d2c2-482a-923f-0676cdfca3eb None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:25.181508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca737-d2c2-482a-923f-0676cdfca3eb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:25.181689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca737-d2c2-482a-923f-0676cdfca3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:25.257248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-866ca737-d2c2-482a-923f-0676cdfca3eb None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:25.258501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-866ca737-d2c2-482a-923f-0676cdfca3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:25.272954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:25.363017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 715/1408] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:25.660906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9424e-90c7-4a3b-9c08-6703a55dcca3 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:25.662735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-57f9424e-90c7-4a3b-9c08-6703a55dcca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:25.826112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9424e-90c7-4a3b-9c08-6703a55dcca3 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:25.911255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:25.967802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d4413-694d-4468-b9de-8baa2e430443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:25.968582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d4413-694d-4468-b9de-8baa2e430443 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:25.968864 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d4413-694d-4468-b9de-8baa2e430443 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:25.969146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d4413-694d-4468-b9de-8baa2e430443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:25.970816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f75d4413-694d-4468-b9de-8baa2e430443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:26.081130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d4413-694d-4468-b9de-8baa2e430443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:26.084429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d4413-694d-4468-b9de-8baa2e430443 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:26.107840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 694/1409] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:35.361178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=38 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:36.105408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=64 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:42.343455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c079-18bf-4287-9a73-efa770235a6e None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:42.344036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c079-18bf-4287-9a73-efa770235a6e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:42.344036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c079-18bf-4287-9a73-efa770235a6e None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:42.345357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 716/1410] 10.222.0.76 () {54 vars in 918 bytes} [Wed Aug 24 17:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:42.350627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800938c-7640-417e-8e56-6d758a0d3edb None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:42.351059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800938c-7640-417e-8e56-6d758a0d3edb None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:42.351172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800938c-7640-417e-8e56-6d758a0d3edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:42.428371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0800938c-7640-417e-8e56-6d758a0d3edb None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:42.429577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0800938c-7640-417e-8e56-6d758a0d3edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:42.536734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 695/1411] 10.222.0.76 () {58 vars in 1015 bytes} [Wed Aug 24 17:42:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:42.552121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81d4e8-a94c-4f2b-83bc-083ff497ba9d None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:42.552408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81d4e8-a94c-4f2b-83bc-083ff497ba9d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:42.552691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81d4e8-a94c-4f2b-83bc-083ff497ba9d None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:42.554065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 717/1412] 10.222.0.76 () {54 vars in 918 bytes} [Wed Aug 24 17:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:42.559174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2ba25-d955-408f-9d56-835303a5fa5f None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:42.559480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2ba25-d955-408f-9d56-835303a5fa5f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:42.559759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2ba25-d955-408f-9d56-835303a5fa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:42.632204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:42.635152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-68e2ba25-d955-408f-9d56-835303a5fa5f None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:42.636889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-68e2ba25-d955-408f-9d56-835303a5fa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:42.752089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 696/1413] 10.222.0.76 () {58 vars in 1015 bytes} [Wed Aug 24 17:42:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:45.196747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6545b-6dfa-44be-8fad-8b2a9b597a6a None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:45.197294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6545b-6dfa-44be-8fad-8b2a9b597a6a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:45.197294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6545b-6dfa-44be-8fad-8b2a9b597a6a None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:45.198561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:45.204172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e99be-b60f-432c-ae60-57d75ddc1527 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:45.204639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e99be-b60f-432c-ae60-57d75ddc1527 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:45.204738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e99be-b60f-432c-ae60-57d75ddc1527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:45.272570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0d0e99be-b60f-432c-ae60-57d75ddc1527 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:45.273813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0d0e99be-b60f-432c-ae60-57d75ddc1527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:45.331345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:45.376374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:45.672345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6545b-6dfa-44be-8fad-8b2a9b597a6a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:45.673906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-07e6545b-6dfa-44be-8fad-8b2a9b597a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:45.837877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6545b-6dfa-44be-8fad-8b2a9b597a6a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:45.911384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:45.955244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6bc02-82b1-45f6-8a59-02e09e675867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:45.955862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6bc02-82b1-45f6-8a59-02e09e675867 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:45.956142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6bc02-82b1-45f6-8a59-02e09e675867 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:45.956438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6bc02-82b1-45f6-8a59-02e09e675867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:45.958146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-dfa6bc02-82b1-45f6-8a59-02e09e675867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:46.054604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6bc02-82b1-45f6-8a59-02e09e675867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:46.056249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6bc02-82b1-45f6-8a59-02e09e675867 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:46.077245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 719/1416] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:46.977791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c9f53-5e49-4603-af0c-66c98a5afe65 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:46.978382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c9f53-5e49-4603-af0c-66c98a5afe65 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:46.978382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c9f53-5e49-4603-af0c-66c98a5afe65 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:46.979758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:46.985605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4ea7-ab1d-4ea5-9f6b-2ee76c855671 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:46.986076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4ea7-ab1d-4ea5-9f6b-2ee76c855671 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:46.986193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4ea7-ab1d-4ea5-9f6b-2ee76c855671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.066683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a65f4ea7-ab1d-4ea5-9f6b-2ee76c855671 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:47.067828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a65f4ea7-ab1d-4ea5-9f6b-2ee76c855671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.110553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:47.200339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:47.350262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c9f53-5e49-4603-af0c-66c98a5afe65 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.351698 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5c3c9f53-5e49-4603-af0c-66c98a5afe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.501784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c9f53-5e49-4603-af0c-66c98a5afe65 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.543057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:47.619208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03768a-a7f7-457b-8377-78ac0a35c651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.619752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03768a-a7f7-457b-8377-78ac0a35c651 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.620032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03768a-a7f7-457b-8377-78ac0a35c651 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.620304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03768a-a7f7-457b-8377-78ac0a35c651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.622252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8c03768a-a7f7-457b-8377-78ac0a35c651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.716621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03768a-a7f7-457b-8377-78ac0a35c651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.718295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03768a-a7f7-457b-8377-78ac0a35c651 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.737763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 699/1419] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:48.625307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545514b-562f-4330-b556-baa52775e5bd None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.625893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545514b-562f-4330-b556-baa52775e5bd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.625893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545514b-562f-4330-b556-baa52775e5bd None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.627294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:48.632484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097ca84-f49c-4f19-b8e2-9132a3bf05f4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.632916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097ca84-f49c-4f19-b8e2-9132a3bf05f4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.633022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097ca84-f49c-4f19-b8e2-9132a3bf05f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.688715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:48.710560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7097ca84-f49c-4f19-b8e2-9132a3bf05f4 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:48.711775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7097ca84-f49c-4f19-b8e2-9132a3bf05f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:48.811893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:48.968425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545514b-562f-4330-b556-baa52775e5bd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:48.969878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0545514b-562f-4330-b556-baa52775e5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:49.125012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545514b-562f-4330-b556-baa52775e5bd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.138346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:49.246542 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e93d1-48a7-4e63-964a-99b89437c28c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.247512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e93d1-48a7-4e63-964a-99b89437c28c service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.247799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e93d1-48a7-4e63-964a-99b89437c28c service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.248073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e93d1-48a7-4e63-964a-99b89437c28c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.249773 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5a9e93d1-48a7-4e63-964a-99b89437c28c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:49.345519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e93d1-48a7-4e63-964a-99b89437c28c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.347142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e93d1-48a7-4e63-964a-99b89437c28c service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.365363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 722/1422] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:50.261555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd294a23-27f5-4e6e-9332-6717c08ec202 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.262221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd294a23-27f5-4e6e-9332-6717c08ec202 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.262221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd294a23-27f5-4e6e-9332-6717c08ec202 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.263497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:50.268587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb46f9-b148-4a50-9068-5bde99b92142 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.269033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb46f9-b148-4a50-9068-5bde99b92142 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.269144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb46f9-b148-4a50-9068-5bde99b92142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.289689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:50.352974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-56cb46f9-b148-4a50-9068-5bde99b92142 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:50.355422 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-56cb46f9-b148-4a50-9068-5bde99b92142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:50.456758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:50.609322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd294a23-27f5-4e6e-9332-6717c08ec202 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.610917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cd294a23-27f5-4e6e-9332-6717c08ec202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:50.728586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:50.767509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd294a23-27f5-4e6e-9332-6717c08ec202 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.885792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49047b74-430c-465b-8577-db8aa25ce77c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.886730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49047b74-430c-465b-8577-db8aa25ce77c service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.887044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49047b74-430c-465b-8577-db8aa25ce77c service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.887322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49047b74-430c-465b-8577-db8aa25ce77c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.889013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-49047b74-430c-465b-8577-db8aa25ce77c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:50.982000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:50.997463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49047b74-430c-465b-8577-db8aa25ce77c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.999282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49047b74-430c-465b-8577-db8aa25ce77c service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:51.018438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 702/1425] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:51.912430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00a97-ec06-45a6-a64a-67caae420389 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:51.912994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00a97-ec06-45a6-a64a-67caae420389 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:51.912994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00a97-ec06-45a6-a64a-67caae420389 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:51.914423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:51.920763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7342f-158f-470b-808a-4598afa6d265 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:51.921199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7342f-158f-470b-808a-4598afa6d265 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:51.921308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7342f-158f-470b-808a-4598afa6d265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:51.998498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f8e7342f-158f-470b-808a-4598afa6d265 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:52.002003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f8e7342f-158f-470b-808a-4598afa6d265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:52.099358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:52.254888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c00a97-ec06-45a6-a64a-67caae420389 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:52.255823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c8c00a97-ec06-45a6-a64a-67caae420389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:52.343580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:52.406975 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c00a97-ec06-45a6-a64a-67caae420389 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:52.525077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26b07c-4979-42d1-aa21-bc212697aa62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:52.525588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26b07c-4979-42d1-aa21-bc212697aa62 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:52.525882 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26b07c-4979-42d1-aa21-bc212697aa62 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:52.526174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26b07c-4979-42d1-aa21-bc212697aa62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:52.528352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-fc26b07c-4979-42d1-aa21-bc212697aa62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:52.578439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:52.620666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26b07c-4979-42d1-aa21-bc212697aa62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:52.622274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26b07c-4979-42d1-aa21-bc212697aa62 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:52.640746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 725/1428] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:53.516892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f727ed-4695-4ad1-87c5-63f7651c9d55 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:53.517476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f727ed-4695-4ad1-87c5-63f7651c9d55 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:53.517476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f727ed-4695-4ad1-87c5-63f7651c9d55 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:53.518832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:53.524071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22492-ae8a-4d82-86a4-e118ba1f2069 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:53.524525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22492-ae8a-4d82-86a4-e118ba1f2069 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:53.524623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22492-ae8a-4d82-86a4-e118ba1f2069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:53.605870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-94a22492-ae8a-4d82-86a4-e118ba1f2069 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:53.607104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-94a22492-ae8a-4d82-86a4-e118ba1f2069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:53.710538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:53.871954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f727ed-4695-4ad1-87c5-63f7651c9d55 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:53.873492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f6f727ed-4695-4ad1-87c5-63f7651c9d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:53.928123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:54.032737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f727ed-4695-4ad1-87c5-63f7651c9d55 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:54.146530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c63ea8-9280-4d42-9c93-638a1447b765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:54.147051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63ea8-9280-4d42-9c93-638a1447b765 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:54.147326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63ea8-9280-4d42-9c93-638a1447b765 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:54.147594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63ea8-9280-4d42-9c93-638a1447b765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:54.149217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-07c63ea8-9280-4d42-9c93-638a1447b765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:54.156891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:54.243404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c63ea8-9280-4d42-9c93-638a1447b765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:54.245036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c63ea8-9280-4d42-9c93-638a1447b765 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:54.264400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 705/1431] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:55.124980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f29a-aeaf-4147-92c8-48b2588952d0 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:55.125517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f29a-aeaf-4147-92c8-48b2588952d0 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:55.125517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f29a-aeaf-4147-92c8-48b2588952d0 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:55.126841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:55.132213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7a8e-75a2-4788-aded-5a08c0b44aa3 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:55.132695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7a8e-75a2-4788-aded-5a08c0b44aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:55.132820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7a8e-75a2-4788-aded-5a08c0b44aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:55.216724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-bbea7a8e-75a2-4788-aded-5a08c0b44aa3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:55.218031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bbea7a8e-75a2-4788-aded-5a08c0b44aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:55.290960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:55.322882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:55.474811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058f29a-aeaf-4147-92c8-48b2588952d0 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:55.476383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2058f29a-aeaf-4147-92c8-48b2588952d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:55.494872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:55.627727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058f29a-aeaf-4147-92c8-48b2588952d0 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:55.717841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:55.746628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b78899-39f1-415e-8359-309faab3a239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:55.747152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b78899-39f1-415e-8359-309faab3a239 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:55.747430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b78899-39f1-415e-8359-309faab3a239 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:55.747718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b78899-39f1-415e-8359-309faab3a239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:55.749339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-99b78899-39f1-415e-8359-309faab3a239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:55.843491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b78899-39f1-415e-8359-309faab3a239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:55.845099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b78899-39f1-415e-8359-309faab3a239 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:55.863769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 728/1434] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:56.748533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc042ec-175c-4146-98a0-2c5bae4b14e3 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:56.749191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc042ec-175c-4146-98a0-2c5bae4b14e3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:56.749191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc042ec-175c-4146-98a0-2c5bae4b14e3 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:56.750479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:56.755529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272dc3-113e-462c-b6fa-14c4b2b3a949 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:56.755958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272dc3-113e-462c-b6fa-14c4b2b3a949 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:56.756079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272dc3-113e-462c-b6fa-14c4b2b3a949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:56.835241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b8272dc3-113e-462c-b6fa-14c4b2b3a949 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:56.836484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b8272dc3-113e-462c-b6fa-14c4b2b3a949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:56.878363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:56.936393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:57.090709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc042ec-175c-4146-98a0-2c5bae4b14e3 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:57.092196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5bc042ec-175c-4146-98a0-2c5bae4b14e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:57.245885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc042ec-175c-4146-98a0-2c5bae4b14e3 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:57.302615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:57.362864 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eaff9-e18e-4ce2-9f74-cdc0f314e9e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:57.363382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eaff9-e18e-4ce2-9f74-cdc0f314e9e4 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:57.363656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eaff9-e18e-4ce2-9f74-cdc0f314e9e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:57.363923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eaff9-e18e-4ce2-9f74-cdc0f314e9e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:57.365521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6f1eaff9-e18e-4ce2-9f74-cdc0f314e9e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:57.461498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1eaff9-e18e-4ce2-9f74-cdc0f314e9e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:57.463163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1eaff9-e18e-4ce2-9f74-cdc0f314e9e4 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:57.483178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 708/1437] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:58.359313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a211d7-09c6-45e8-b6f3-ca20c31230a7 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:58.359887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a211d7-09c6-45e8-b6f3-ca20c31230a7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:58.359887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a211d7-09c6-45e8-b6f3-ca20c31230a7 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:58.361270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:58.366491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e64f-4d2e-4bb7-8dcd-46d8d855c438 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:58.366887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e64f-4d2e-4bb7-8dcd-46d8d855c438 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:58.367038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e64f-4d2e-4bb7-8dcd-46d8d855c438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:58.446296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0819e64f-4d2e-4bb7-8dcd-46d8d855c438 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:58.447524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0819e64f-4d2e-4bb7-8dcd-46d8d855c438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:58.450338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:58.547036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:58.697361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a211d7-09c6-45e8-b6f3-ca20c31230a7 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:58.699622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-94a211d7-09c6-45e8-b6f3-ca20c31230a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:58.850213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a211d7-09c6-45e8-b6f3-ca20c31230a7 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:58.879250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:58.969231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650dd7e-a230-411d-87fd-5a40818ea6e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:58.969839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650dd7e-a230-411d-87fd-5a40818ea6e0 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:58.970144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650dd7e-a230-411d-87fd-5a40818ea6e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:58.970413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650dd7e-a230-411d-87fd-5a40818ea6e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:58.972015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b650dd7e-a230-411d-87fd-5a40818ea6e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:59.062912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650dd7e-a230-411d-87fd-5a40818ea6e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:59.064593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650dd7e-a230-411d-87fd-5a40818ea6e0 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:59.082797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 731/1440] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:00.961649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f79988-ae05-4084-8e3b-9dad58f638a2 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:00.962283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f79988-ae05-4084-8e3b-9dad58f638a2 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:00.962283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f79988-ae05-4084-8e3b-9dad58f638a2 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:00.964269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:00.970162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199ad1-6c5e-41ce-ad35-fc48263acea2 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:00.970677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199ad1-6c5e-41ce-ad35-fc48263acea2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:00.970783 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199ad1-6c5e-41ce-ad35-fc48263acea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.013781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:01.042680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7b199ad1-6c5e-41ce-ad35-fc48263acea2 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:01.044129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7b199ad1-6c5e-41ce-ad35-fc48263acea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:01.143479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:01.433984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f79988-ae05-4084-8e3b-9dad58f638a2 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:01.435584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-30f79988-ae05-4084-8e3b-9dad58f638a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:01.541255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f79988-ae05-4084-8e3b-9dad58f638a2 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.575102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:01.616289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d7f74-bea9-4d00-954f-01a456838f37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.616885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d7f74-bea9-4d00-954f-01a456838f37 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.617160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d7f74-bea9-4d00-954f-01a456838f37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.617422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d7f74-bea9-4d00-954f-01a456838f37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.619054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cd1d7f74-bea9-4d00-954f-01a456838f37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:01.728504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d7f74-bea9-4d00-954f-01a456838f37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.730073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d7f74-bea9-4d00-954f-01a456838f37 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.749115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 711/1443] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:03.825619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a5982-2f9f-4b74-8cf3-7d80a610521f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:03.826212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a5982-2f9f-4b74-8cf3-7d80a610521f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:03.826212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a5982-2f9f-4b74-8cf3-7d80a610521f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:03.827583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:03.833034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d2e91-a746-40a2-b0c2-7db0bdfcdce1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:03.833471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d2e91-a746-40a2-b0c2-7db0bdfcdce1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:03.833662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d2e91-a746-40a2-b0c2-7db0bdfcdce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:03.899881 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ab1d2e91-a746-40a2-b0c2-7db0bdfcdce1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:03.901670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ab1d2e91-a746-40a2-b0c2-7db0bdfcdce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:03.915104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:04.001106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:04.289328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a5982-2f9f-4b74-8cf3-7d80a610521f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:04.290896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4b6a5982-2f9f-4b74-8cf3-7d80a610521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:04.395641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a5982-2f9f-4b74-8cf3-7d80a610521f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:04.408611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:04.469835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb06532-ac99-49a5-a425-5f96d6dee6ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:04.470697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb06532-ac99-49a5-a425-5f96d6dee6ff service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:04.470697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb06532-ac99-49a5-a425-5f96d6dee6ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:04.471086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb06532-ac99-49a5-a425-5f96d6dee6ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:04.472561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5bb06532-ac99-49a5-a425-5f96d6dee6ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:04.582361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb06532-ac99-49a5-a425-5f96d6dee6ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:04.583725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb06532-ac99-49a5-a425-5f96d6dee6ff service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:04.602235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 734/1446] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:06.910519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a7ef1-d3d4-4ebd-b418-f3f13ca6af21 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:06.911081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a7ef1-d3d4-4ebd-b418-f3f13ca6af21 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:06.911081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a7ef1-d3d4-4ebd-b418-f3f13ca6af21 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:06.912443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:06.919130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ddfac-28ea-4469-9088-b84fe137a6a2 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:06.919130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ddfac-28ea-4469-9088-b84fe137a6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:06.919579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ddfac-28ea-4469-9088-b84fe137a6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:06.968010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:06.986297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ff2ddfac-28ea-4469-9088-b84fe137a6a2 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:06.991676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ff2ddfac-28ea-4469-9088-b84fe137a6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:07.091253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:07.380914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a7ef1-d3d4-4ebd-b418-f3f13ca6af21 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:07.382486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8e9a7ef1-d3d4-4ebd-b418-f3f13ca6af21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:07.487158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a7ef1-d3d4-4ebd-b418-f3f13ca6af21 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:07.530690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:07.566705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693663-621d-4d11-8c37-e96c52abd633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:07.567319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693663-621d-4d11-8c37-e96c52abd633 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:07.567619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693663-621d-4d11-8c37-e96c52abd633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:07.567876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693663-621d-4d11-8c37-e96c52abd633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:07.569999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3d693663-621d-4d11-8c37-e96c52abd633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:07.682748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d693663-621d-4d11-8c37-e96c52abd633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:07.684193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d693663-621d-4d11-8c37-e96c52abd633 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:07.703573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 714/1449] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:09.753303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82412-1983-4df2-85ec-592912413d53 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:09.753941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82412-1983-4df2-85ec-592912413d53 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:09.753941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82412-1983-4df2-85ec-592912413d53 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:09.755312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:09.761414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fabef-87e4-425b-b9b0-491a1777ea3b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:09.761890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fabef-87e4-425b-b9b0-491a1777ea3b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:09.762010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fabef-87e4-425b-b9b0-491a1777ea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:09.829828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-196fabef-87e4-425b-b9b0-491a1777ea3b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:09.831095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-196fabef-87e4-425b-b9b0-491a1777ea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:09.851640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:09.930076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:10.222628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d82412-1983-4df2-85ec-592912413d53 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:10.224207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-74d82412-1983-4df2-85ec-592912413d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:10.450366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d82412-1983-4df2-85ec-592912413d53 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:10.470702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:10.523707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e7eb9-0490-48c3-9c48-ae7efa9e0fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:10.524489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e7eb9-0490-48c3-9c48-ae7efa9e0fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:10.524976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e7eb9-0490-48c3-9c48-ae7efa9e0fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:10.526294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e7eb9-0490-48c3-9c48-ae7efa9e0fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:10.528125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cf9e7eb9-0490-48c3-9c48-ae7efa9e0fb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:10.637842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e7eb9-0490-48c3-9c48-ae7efa9e0fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:10.639319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e7eb9-0490-48c3-9c48-ae7efa9e0fb4 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:10.657766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 737/1452] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:12.659419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba40b1f-feb4-4db1-bb20-ee4a228ccfc0 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:12.659949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba40b1f-feb4-4db1-bb20-ee4a228ccfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:12.659949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba40b1f-feb4-4db1-bb20-ee4a228ccfc0 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:12.661243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:12.666845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e02fe-3668-42c1-9cca-c9a3ae61c20c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:12.667313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e02fe-3668-42c1-9cca-c9a3ae61c20c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:12.667414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e02fe-3668-42c1-9cca-c9a3ae61c20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:12.732817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:12.736201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-683e02fe-3668-42c1-9cca-c9a3ae61c20c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:12.737402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-683e02fe-3668-42c1-9cca-c9a3ae61c20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:12.839068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:43:13.123605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba40b1f-feb4-4db1-bb20-ee4a228ccfc0 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:13.125174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1ba40b1f-feb4-4db1-bb20-ee4a228ccfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:13.231875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba40b1f-feb4-4db1-bb20-ee4a228ccfc0 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:13.281468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:13.307579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee4081-3768-45c3-a0e0-a25a2115f722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:13.308172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee4081-3768-45c3-a0e0-a25a2115f722 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:13.308460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee4081-3768-45c3-a0e0-a25a2115f722 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:13.308757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee4081-3768-45c3-a0e0-a25a2115f722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:13.310420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7bee4081-3768-45c3-a0e0-a25a2115f722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:13.421308 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee4081-3768-45c3-a0e0-a25a2115f722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:13.422784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee4081-3768-45c3-a0e0-a25a2115f722 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:13.442190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 717/1455] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:15.543039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe2811-acd2-467d-a253-679fb978eb71 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:15.543858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe2811-acd2-467d-a253-679fb978eb71 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:15.543858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe2811-acd2-467d-a253-679fb978eb71 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:15.545224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:15.550515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd1b3-d7b9-466c-9158-a3d73214eab7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:15.550976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd1b3-d7b9-466c-9158-a3d73214eab7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:15.551073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd1b3-d7b9-466c-9158-a3d73214eab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:15.618097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d34fd1b3-d7b9-466c-9158-a3d73214eab7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:15.619325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d34fd1b3-d7b9-466c-9158-a3d73214eab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:15.649271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:15.718758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:16.005802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe2811-acd2-467d-a253-679fb978eb71 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:16.007362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-16fe2811-acd2-467d-a253-679fb978eb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:16.112996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe2811-acd2-467d-a253-679fb978eb71 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:16.141497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:16.188341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac1239-6f1b-4828-b7ed-c6c79a8d918f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:16.188863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1239-6f1b-4828-b7ed-c6c79a8d918f service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:16.189139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1239-6f1b-4828-b7ed-c6c79a8d918f service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:16.189407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1239-6f1b-4828-b7ed-c6c79a8d918f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:16.191039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2cac1239-6f1b-4828-b7ed-c6c79a8d918f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:16.299124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac1239-6f1b-4828-b7ed-c6c79a8d918f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:16.300589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac1239-6f1b-4828-b7ed-c6c79a8d918f service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:16.318989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 740/1458] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:18.315016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f40c7-f442-433c-85c4-cee2428b5ec7 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:18.315586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f40c7-f442-433c-85c4-cee2428b5ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:18.315586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f40c7-f442-433c-85c4-cee2428b5ec7 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:18.316932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:18.322990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edb99-4bdc-4dac-815f-b87d13a0e629 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:18.323442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edb99-4bdc-4dac-815f-b87d13a0e629 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:18.323542 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edb99-4bdc-4dac-815f-b87d13a0e629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:18.389564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-190edb99-4bdc-4dac-815f-b87d13a0e629 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:18.390837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-190edb99-4bdc-4dac-815f-b87d13a0e629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:18.399421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:18.489489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:18.788997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f40c7-f442-433c-85c4-cee2428b5ec7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:18.790616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-254f40c7-f442-433c-85c4-cee2428b5ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:18.898053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f40c7-f442-433c-85c4-cee2428b5ec7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:18.954932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:18.973453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83050747-740c-4c68-bb83-a2448b76e611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:18.974091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83050747-740c-4c68-bb83-a2448b76e611 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:18.974369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83050747-740c-4c68-bb83-a2448b76e611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:18.974637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83050747-740c-4c68-bb83-a2448b76e611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:18.976246 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-83050747-740c-4c68-bb83-a2448b76e611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:19.085948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83050747-740c-4c68-bb83-a2448b76e611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:19.087334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83050747-740c-4c68-bb83-a2448b76e611 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:19.109150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 720/1461] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:21.091926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54b5a-e75f-4117-9b8a-d25a9baec3b5 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:21.092508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54b5a-e75f-4117-9b8a-d25a9baec3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:21.092508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54b5a-e75f-4117-9b8a-d25a9baec3b5 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:21.093823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:21.099440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9da15-7828-4b64-98f5-284dcaf46cfe None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:21.099858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9da15-7828-4b64-98f5-284dcaf46cfe None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:21.099993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9da15-7828-4b64-98f5-284dcaf46cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:21.171696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7df9da15-7828-4b64-98f5-284dcaf46cfe None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:21.172958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7df9da15-7828-4b64-98f5-284dcaf46cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:21.212151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:21.271794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:21.562299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54b5a-e75f-4117-9b8a-d25a9baec3b5 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:21.563888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-edd54b5a-e75f-4117-9b8a-d25a9baec3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:21.671078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54b5a-e75f-4117-9b8a-d25a9baec3b5 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:21.708197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:21.745039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea5613-240f-49f5-b5a5-06caa1c5cb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:21.745637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5613-240f-49f5-b5a5-06caa1c5cb88 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:21.745942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5613-240f-49f5-b5a5-06caa1c5cb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:21.746210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5613-240f-49f5-b5a5-06caa1c5cb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:21.747802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d6ea5613-240f-49f5-b5a5-06caa1c5cb88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:21.858001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea5613-240f-49f5-b5a5-06caa1c5cb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:21.859344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea5613-240f-49f5-b5a5-06caa1c5cb88 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:21.877538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 743/1464] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:23.818783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909443a-4a46-4d26-a3d3-4f5fe6633bef None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:23.819330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909443a-4a46-4d26-a3d3-4f5fe6633bef None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:23.819330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909443a-4a46-4d26-a3d3-4f5fe6633bef None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:23.820699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:23.826345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5f55f-2673-4640-a46d-2d8afc38b759 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:23.826767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5f55f-2673-4640-a46d-2d8afc38b759 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:23.826872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5f55f-2673-4640-a46d-2d8afc38b759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:23.892521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-52d5f55f-2673-4640-a46d-2d8afc38b759 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:23.893715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-52d5f55f-2673-4640-a46d-2d8afc38b759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:23.911467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:23.991943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:24.275492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909443a-4a46-4d26-a3d3-4f5fe6633bef None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:24.277135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8909443a-4a46-4d26-a3d3-4f5fe6633bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:24.380971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909443a-4a46-4d26-a3d3-4f5fe6633bef None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:24.448829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:24.455486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1769e2-1a4c-46a2-aec5-858f12f9130c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:24.456090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1769e2-1a4c-46a2-aec5-858f12f9130c service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:24.456367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1769e2-1a4c-46a2-aec5-858f12f9130c service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:24.456634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1769e2-1a4c-46a2-aec5-858f12f9130c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:24.458266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2e1769e2-1a4c-46a2-aec5-858f12f9130c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:24.568316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1769e2-1a4c-46a2-aec5-858f12f9130c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:24.569643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1769e2-1a4c-46a2-aec5-858f12f9130c service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:24.591158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 723/1467] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:26.529331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fd33f-aa67-467f-8279-eda2fc63f7e4 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:26.529958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fd33f-aa67-467f-8279-eda2fc63f7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:26.529958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fd33f-aa67-467f-8279-eda2fc63f7e4 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:26.531328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:26.536594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b868b-0553-4743-ad58-460eb6a9c240 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:26.537007 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b868b-0553-4743-ad58-460eb6a9c240 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:26.537152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b868b-0553-4743-ad58-460eb6a9c240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:26.606285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6d5b868b-0553-4743-ad58-460eb6a9c240 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:26.607514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6d5b868b-0553-4743-ad58-460eb6a9c240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:26.656388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:26.705508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:26.986072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fd33f-aa67-467f-8279-eda2fc63f7e4 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:26.987611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-802fd33f-aa67-467f-8279-eda2fc63f7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:27.102236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fd33f-aa67-467f-8279-eda2fc63f7e4 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:27.153977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:27.180166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca957ddb-cc14-442c-b63e-1e17ffc4c8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:27.180767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca957ddb-cc14-442c-b63e-1e17ffc4c8ba service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:27.181646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca957ddb-cc14-442c-b63e-1e17ffc4c8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:27.181955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca957ddb-cc14-442c-b63e-1e17ffc4c8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:27.183562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ca957ddb-cc14-442c-b63e-1e17ffc4c8ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:27.293870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca957ddb-cc14-442c-b63e-1e17ffc4c8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:27.295313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca957ddb-cc14-442c-b63e-1e17ffc4c8ba service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:27.314637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 746/1470] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:36.703725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=22 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:37.312023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=66 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:47.147807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56227232-45bf-464e-9fb2-47f35a9e4b6a None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:47.148359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56227232-45bf-464e-9fb2-47f35a9e4b6a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:47.148359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56227232-45bf-464e-9fb2-47f35a9e4b6a None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:47.150364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 725/1471] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:47.156459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4861d1c-81ed-4de7-83b4-ab938708b790 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:47.157032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4861d1c-81ed-4de7-83b4-ab938708b790 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:47.157032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4861d1c-81ed-4de7-83b4-ab938708b790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:47.224049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d4861d1c-81ed-4de7-83b4-ab938708b790 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:47.225278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d4861d1c-81ed-4de7-83b4-ab938708b790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:47.325395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 747/1472] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:47.626574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42893f-d603-4327-9c86-ad9ad6bb2338 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:47.626858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42893f-d603-4327-9c86-ad9ad6bb2338 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:47.627135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42893f-d603-4327-9c86-ad9ad6bb2338 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:47.629034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 726/1473] 10.222.0.76 () {54 vars in 967 bytes} [Wed Aug 24 17:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:47.634177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a80a-1670-40d4-8379-9341f7b9ba1d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:47.636623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a80a-1670-40d4-8379-9341f7b9ba1d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:47.636623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a80a-1670-40d4-8379-9341f7b9ba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:47.728333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-79a3a80a-1670-40d4-8379-9341f7b9ba1d None None] MFA Rules not processed for user `b4d874536170494496769b4f37234b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:47.729593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-79a3a80a-1670-40d4-8379-9341f7b9ba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:47.753216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:47.822065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 748/1474] 10.222.0.76 () {58 vars in 1064 bytes} [Wed Aug 24 17:43:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:47.830199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42893f-d603-4327-9c86-ad9ad6bb2338 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:47.831794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-db42893f-d603-4327-9c86-ad9ad6bb2338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:47.944268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42893f-d603-4327-9c86-ad9ad6bb2338 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:48.027049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bd2e8-bdc2-45ad-8bcd-244813b065c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d874536170494496769b4f37234b2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:48.027661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bd2e8-bdc2-45ad-8bcd-244813b065c0 service glance] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:48.027943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bd2e8-bdc2-45ad-8bcd-244813b065c0 service glance] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:48.028217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bd2e8-bdc2-45ad-8bcd-244813b065c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:48.029843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1e5bd2e8-bdc2-45ad-8bcd-244813b065c0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:48.088181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:48.146634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bd2e8-bdc2-45ad-8bcd-244813b065c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:48.148077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bd2e8-bdc2-45ad-8bcd-244813b065c0 service glance] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:48.169609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 727/1475] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:50.203817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1a30-f1cf-42f8-bdde-86e799920270 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:50.204376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1a30-f1cf-42f8-bdde-86e799920270 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:50.204376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1a30-f1cf-42f8-bdde-86e799920270 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:50.205749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 749/1476] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:50.212266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7524207-cfdf-4315-bb25-90629d479ba6 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:50.213030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7524207-cfdf-4315-bb25-90629d479ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:50.213244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7524207-cfdf-4315-bb25-90629d479ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:50.296194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e7524207-cfdf-4315-bb25-90629d479ba6 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:50.297587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e7524207-cfdf-4315-bb25-90629d479ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:50.422603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 728/1477] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:50.710231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f1a30-f1cf-42f8-bdde-86e799920270 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:50.711835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8e1f1a30-f1cf-42f8-bdde-86e799920270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:50.849009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f1a30-f1cf-42f8-bdde-86e799920270 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:50.898169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:50.947853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c0d9c-7964-4d7d-b92c-313fbae5506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:50.948462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c0d9c-7964-4d7d-b92c-313fbae5506c admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:50.948742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c0d9c-7964-4d7d-b92c-313fbae5506c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:50.949014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c0d9c-7964-4d7d-b92c-313fbae5506c admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:50.949672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c0d9c-7964-4d7d-b92c-313fbae5506c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:50.951484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING py.warnings [None req-d40c0d9c-7964-4d7d-b92c-313fbae5506c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:43:50.951484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: warnings.warn(msg) Aug 24 17:43:50.951484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: Aug 24 17:43:50.952622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c0d9c-7964-4d7d-b92c-313fbae5506c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:50.972555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 750/1478] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 17:43:50 2022] GET /identity/v3/projects => generated 2553 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:43:52.956142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44862447-79c6-4ee3-bf20-3a4edc9c9361 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:52.956717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44862447-79c6-4ee3-bf20-3a4edc9c9361 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:52.956717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44862447-79c6-4ee3-bf20-3a4edc9c9361 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:52.958063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 729/1479] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:52.963642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0415-c425-4111-937d-b1caf30ba8ab None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:52.964145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0415-c425-4111-937d-b1caf30ba8ab None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:52.964243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0415-c425-4111-937d-b1caf30ba8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:53.038510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c62e0415-c425-4111-937d-b1caf30ba8ab None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:53.040275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c62e0415-c425-4111-937d-b1caf30ba8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:53.153197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 751/1480] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:53.458819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44862447-79c6-4ee3-bf20-3a4edc9c9361 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:53.460398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-44862447-79c6-4ee3-bf20-3a4edc9c9361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:53.471121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:53.613584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44862447-79c6-4ee3-bf20-3a4edc9c9361 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:53.699383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:53.738255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118c04d-0898-4162-999a-0977ed199745 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:53.739127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118c04d-0898-4162-999a-0977ed199745 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:53.739660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118c04d-0898-4162-999a-0977ed199745 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:53.740562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118c04d-0898-4162-999a-0977ed199745 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:53.742480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e118c04d-0898-4162-999a-0977ed199745 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:53.847292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118c04d-0898-4162-999a-0977ed199745 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:53.848932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118c04d-0898-4162-999a-0977ed199745 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:53.868867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 730/1481] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:56.131417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e187d24-7dfe-4988-b4b2-c201ee4c1f1a None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.132042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e187d24-7dfe-4988-b4b2-c201ee4c1f1a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.132042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e187d24-7dfe-4988-b4b2-c201ee4c1f1a None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.134190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 752/1482] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:43:56.140301 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58978173-8988-4bc8-a559-3ffb00e5c255 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.142014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58978173-8988-4bc8-a559-3ffb00e5c255 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.142014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58978173-8988-4bc8-a559-3ffb00e5c255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.216250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-58978173-8988-4bc8-a559-3ffb00e5c255 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:56.217482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-58978173-8988-4bc8-a559-3ffb00e5c255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:56.268622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:56.325547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 731/1483] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:56.616793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e187d24-7dfe-4988-b4b2-c201ee4c1f1a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:56.619554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0e187d24-7dfe-4988-b4b2-c201ee4c1f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:56.645433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:56.733767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e187d24-7dfe-4988-b4b2-c201ee4c1f1a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:56.815176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf5cff-2ab7-44df-8aaa-3f3cc726df15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:56.815806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5cff-2ab7-44df-8aaa-3f3cc726df15 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.816083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5cff-2ab7-44df-8aaa-3f3cc726df15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.816353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5cff-2ab7-44df-8aaa-3f3cc726df15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.818236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-28bf5cff-2ab7-44df-8aaa-3f3cc726df15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:56.928185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:56.975972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf5cff-2ab7-44df-8aaa-3f3cc726df15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:56.978006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf5cff-2ab7-44df-8aaa-3f3cc726df15 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:57.000992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 753/1484] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:59.002995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7e3be-d946-4215-8c19-43fa8e573f4e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:59.010224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7e3be-d946-4215-8c19-43fa8e573f4e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:59.010224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7e3be-d946-4215-8c19-43fa8e573f4e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:59.010224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 732/1485] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:59.011295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9b7cb-96c7-48e4-b9ff-07c5539b7d42 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:59.011689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9b7cb-96c7-48e4-b9ff-07c5539b7d42 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:59.011878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9b7cb-96c7-48e4-b9ff-07c5539b7d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:59.080851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f1f9b7cb-96c7-48e4-b9ff-07c5539b7d42 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:59.082044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f1f9b7cb-96c7-48e4-b9ff-07c5539b7d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:59.182534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 754/1486] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:59.472892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7e3be-d946-4215-8c19-43fa8e573f4e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:59.474502 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6df7e3be-d946-4215-8c19-43fa8e573f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:59.582022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7e3be-d946-4215-8c19-43fa8e573f4e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:59.658908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:59.660732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090ebf0-879a-4245-bbd7-4d751ba5c083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:59.661344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ebf0-879a-4245-bbd7-4d751ba5c083 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:59.661619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ebf0-879a-4245-bbd7-4d751ba5c083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:59.661915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ebf0-879a-4245-bbd7-4d751ba5c083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:59.663550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1090ebf0-879a-4245-bbd7-4d751ba5c083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:59.783564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090ebf0-879a-4245-bbd7-4d751ba5c083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:59.784936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090ebf0-879a-4245-bbd7-4d751ba5c083 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:59.806511 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 733/1487] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:01.856128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9760-d7ee-4951-ab78-9bb48776e501 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.856678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9760-d7ee-4951-ab78-9bb48776e501 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.856678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9760-d7ee-4951-ab78-9bb48776e501 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.858069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 755/1488] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:44:01.863379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7435cc-2695-4031-bf9c-bc3ade255820 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.863780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7435cc-2695-4031-bf9c-bc3ade255820 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.863941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7435cc-2695-4031-bf9c-bc3ade255820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.932936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2d7435cc-2695-4031-bf9c-bc3ade255820 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:01.934257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2d7435cc-2695-4031-bf9c-bc3ade255820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:01.989315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:02.034772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 734/1489] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:02.330129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b9760-d7ee-4951-ab78-9bb48776e501 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:02.331690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e14b9760-d7ee-4951-ab78-9bb48776e501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:02.378068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:02.453026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b9760-d7ee-4951-ab78-9bb48776e501 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:02.529759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485adfbb-1b22-4735-95de-076530db829a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:02.530387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485adfbb-1b22-4735-95de-076530db829a service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:02.530662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485adfbb-1b22-4735-95de-076530db829a service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:02.530935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485adfbb-1b22-4735-95de-076530db829a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:02.532531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-485adfbb-1b22-4735-95de-076530db829a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:02.618177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:02.649889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485adfbb-1b22-4735-95de-076530db829a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:02.651417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485adfbb-1b22-4735-95de-076530db829a service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:02.670898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 756/1490] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:04.918680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1bdc-aeac-4e9f-9120-ef979ea1fa08 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:04.919082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1bdc-aeac-4e9f-9120-ef979ea1fa08 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:04.919236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1bdc-aeac-4e9f-9120-ef979ea1fa08 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:04.920609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 735/1491] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 17:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:44:04.926072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ad79-1bcc-40f0-8b65-c57d3f8cc852 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:04.926633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ad79-1bcc-40f0-8b65-c57d3f8cc852 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:04.926633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ad79-1bcc-40f0-8b65-c57d3f8cc852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:04.995256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a735ad79-1bcc-40f0-8b65-c57d3f8cc852 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:04.996498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a735ad79-1bcc-40f0-8b65-c57d3f8cc852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:05.095369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 757/1492] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 17:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:05.383180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1bdc-aeac-4e9f-9120-ef979ea1fa08 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:05.384747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-74be1bdc-aeac-4e9f-9120-ef979ea1fa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:05.498084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1bdc-aeac-4e9f-9120-ef979ea1fa08 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:05.693937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6feefd8-dcb9-448e-9e96-f6f82aafb98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:05.694469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feefd8-dcb9-448e-9e96-f6f82aafb98c service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:05.694806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feefd8-dcb9-448e-9e96-f6f82aafb98c service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:05.695167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feefd8-dcb9-448e-9e96-f6f82aafb98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:05.697010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e6feefd8-dcb9-448e-9e96-f6f82aafb98c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:05.705783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:05.807796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feefd8-dcb9-448e-9e96-f6f82aafb98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:05.809244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feefd8-dcb9-448e-9e96-f6f82aafb98c service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:05.829541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 736/1493] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:15.093518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=82 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:15.827199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=50 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:34.105883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff9e7-781e-4041-8e65-a1c4a0bd3835 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:34.106520 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff9e7-781e-4041-8e65-a1c4a0bd3835 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:34.106520 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff9e7-781e-4041-8e65-a1c4a0bd3835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:34.194118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d44ff9e7-781e-4041-8e65-a1c4a0bd3835 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:34.195303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d44ff9e7-781e-4041-8e65-a1c4a0bd3835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:34.311548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 758/1494] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:45:34.320608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac4914-d349-4b73-8faa-aadf8cd4cb21 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:34.321223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac4914-d349-4b73-8faa-aadf8cd4cb21 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:34.321223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac4914-d349-4b73-8faa-aadf8cd4cb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:34.407215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-61ac4914-d349-4b73-8faa-aadf8cd4cb21 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:34.408460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-61ac4914-d349-4b73-8faa-aadf8cd4cb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:34.510871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 737/1495] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:45:34.515845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ff9e7-781e-4041-8e65-a1c4a0bd3835 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:34.517074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d44ff9e7-781e-4041-8e65-a1c4a0bd3835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:34.531431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ff9e7-781e-4041-8e65-a1c4a0bd3835 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:34.602340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:34.637757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:34.638400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:34.638702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:34.638974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:34.639650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:45:34.640012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:34.641059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING py.warnings [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:45:34.641059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: warnings.warn(msg) Aug 24 17:45:34.641059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: Aug 24 17:45:34.641971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:34.653905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 759/1496] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:45:34.663402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac4914-d349-4b73-8faa-aadf8cd4cb21 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:34.673346 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac4914-d349-4b73-8faa-aadf8cd4cb21 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:34.764154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:34.831954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:34.834085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:34.834582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:34.835067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:34.836258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:34.839669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING py.warnings [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:45:34.839669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: warnings.warn(msg) Aug 24 17:45:34.839669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: Aug 24 17:45:34.842017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:35.018441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 738/1497] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:45:34 2022] POST /identity/v3/projects => generated 379 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:45:35.023361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:35.024954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:35.066055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484ff5b-0de6-41b6-83ec-3a214d499aaa admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:35.186299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:35.187008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:35.187283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:35.187589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:35.188271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:35.189371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING py.warnings [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:45:35.189371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: warnings.warn(msg) Aug 24 17:45:35.189371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: Aug 24 17:45:35.190860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:35.301614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 760/1498] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:45:35 2022] POST /identity/v3/users => generated 290 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:45:35.305828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:35.307542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:35.404441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ebb4b-c580-4b9d-a9a9-fbc879ff1cac admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:35.540184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:35.553226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:35.554072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:35.554570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:35.555022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:35.556147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:35.558038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING py.warnings [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:45:35.558038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: warnings.warn(msg) Aug 24 17:45:35.558038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: Aug 24 17:45:35.559621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:35.579126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 739/1499] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:45:35.583690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:35.585609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:35.599422 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b67cf-3bb1-4ab4-8539-c4a2082a137f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:35.630247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:35.734322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515c887-8bb7-46fe-97bc-1028f0e4b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:35.735094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c887-8bb7-46fe-97bc-1028f0e4b01b admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:35.735424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c887-8bb7-46fe-97bc-1028f0e4b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:35.735744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c887-8bb7-46fe-97bc-1028f0e4b01b admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:35.736458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515c887-8bb7-46fe-97bc-1028f0e4b01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:35.737724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING py.warnings [None req-2515c887-8bb7-46fe-97bc-1028f0e4b01b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:45:35.737724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: warnings.warn(msg) Aug 24 17:45:35.737724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: Aug 24 17:45:35.739373 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515c887-8bb7-46fe-97bc-1028f0e4b01b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:35.753647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 761/1500] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:45:35.758575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:35.760156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:35.809358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031cc0a-8a7f-4692-ae43-abf5d39f24f2 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:35.922878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6b63e-931a-4f23-b40c-5793cbbd597d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:35.923544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b63e-931a-4f23-b40c-5793cbbd597d admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:35.923870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b63e-931a-4f23-b40c-5793cbbd597d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:35.925135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b63e-931a-4f23-b40c-5793cbbd597d admin admin] PATH_INFO: `/v3/projects/1024982ef1a64b6b9230b15d7381280f/users/10f6964277a24237a1798b072705f408/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:35.949070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6b63e-931a-4f23-b40c-5793cbbd597d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1024982ef1a64b6b9230b15d7381280f, user_id=10f6964277a24237a1798b072705f408, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:35.950193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING py.warnings [None req-0be6b63e-931a-4f23-b40c-5793cbbd597d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:45:35.950193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: warnings.warn(msg) Aug 24 17:45:35.950193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: Aug 24 17:45:35.952129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6b63e-931a-4f23-b40c-5793cbbd597d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:36.052568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 740/1501] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:45:35 2022] PUT /identity/v3/projects/1024982ef1a64b6b9230b15d7381280f/users/10f6964277a24237a1798b072705f408/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:45:36.066230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021981b-fb9f-4074-bf77-f061726263a7 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:36.066527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021981b-fb9f-4074-bf77-f061726263a7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:36.066694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021981b-fb9f-4074-bf77-f061726263a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:36.135976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-6021981b-fb9f-4074-bf77-f061726263a7 None None] MFA Rules not processed for user `10f6964277a24237a1798b072705f408`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:36.137407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6021981b-fb9f-4074-bf77-f061726263a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:36.249938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 762/1502] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:45:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:45:36.265011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b848128-bcc4-4c5a-a772-02b9d43de767 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:36.265284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b848128-bcc4-4c5a-a772-02b9d43de767 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:36.265615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b848128-bcc4-4c5a-a772-02b9d43de767 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:36.267096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 741/1503] 10.222.0.76 () {54 vars in 964 bytes} [Wed Aug 24 17:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:45:36.274546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d85a3-209c-4fbd-9f41-3384fe2bfa2a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:36.274886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d85a3-209c-4fbd-9f41-3384fe2bfa2a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:36.275203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d85a3-209c-4fbd-9f41-3384fe2bfa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:36.317518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:36.387865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d41d85a3-209c-4fbd-9f41-3384fe2bfa2a None None] MFA Rules not processed for user `6c7a064cf91243648c6cc6d6ca576505`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:36.389118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d41d85a3-209c-4fbd-9f41-3384fe2bfa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:36.499228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 763/1504] 10.222.0.76 () {58 vars in 1061 bytes} [Wed Aug 24 17:45:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:45:36.507768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b848128-bcc4-4c5a-a772-02b9d43de767 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:36.509319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9b848128-bcc4-4c5a-a772-02b9d43de767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:36.570170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:36.648168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b848128-bcc4-4c5a-a772-02b9d43de767 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:36.731137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:36.731742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:36.732021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:36.732290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:36.733977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:36.843392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:36.844771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:36.849509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:36.867296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 742/1505] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:36.921661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d0eba-d43d-4ec4-9804-0169fee7191c None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:36.921978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d0eba-d43d-4ec4-9804-0169fee7191c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:36.922347 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d0eba-d43d-4ec4-9804-0169fee7191c None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:36.923660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 764/1506] 10.222.0.76 () {52 vars in 830 bytes} [Wed Aug 24 17:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:45:36.927834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:36.929100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:37.044358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76523450-4220-4c6b-910e-c51f1e5c71d8 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:37.152140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:37.152714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:37.152998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:37.153278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] PATH_INFO: `/v3/users/10f6964277a24237a1798b072705f408` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:37.168357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] RBAC: Authorizing `identity:delete_user(user_id=10f6964277a24237a1798b072705f408)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:37.169418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING py.warnings [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:45:37.169418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: warnings.warn(msg) Aug 24 17:45:37.169418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: Aug 24 17:45:37.170692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:37.182971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:37.481093 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10f6964277a24237a1798b072705f408'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:37.554779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10f6964277a24237a1798b072705f408'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:37.555441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10f6964277a24237a1798b072705f408'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:37.573985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10f6964277a24237a1798b072705f408'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:37.574619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10f6964277a24237a1798b072705f408'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:37.608477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 743/1507] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:45:36 2022] DELETE /identity/v3/users/10f6964277a24237a1798b072705f408 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:45:37.613823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de48cdb-f21d-4cd7-a3da-c220bc2ef0ee None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:37.614564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de48cdb-f21d-4cd7-a3da-c220bc2ef0ee None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:37.614871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de48cdb-f21d-4cd7-a3da-c220bc2ef0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:37.680046 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7de48cdb-f21d-4cd7-a3da-c220bc2ef0ee None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:37.681287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7de48cdb-f21d-4cd7-a3da-c220bc2ef0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:37.765948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:37.778792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 765/1508] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:45:37.794558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:37.795817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:37.919065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266cfd7-024b-4c9a-a84d-a32f8db8832d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:37.981327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:38.002488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ee6ca-526e-4a8f-921d-bf5cd14e9c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:38.003108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ee6ca-526e-4a8f-921d-bf5cd14e9c5a service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:38.003395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ee6ca-526e-4a8f-921d-bf5cd14e9c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:38.003671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ee6ca-526e-4a8f-921d-bf5cd14e9c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:38.005556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-637ee6ca-526e-4a8f-921d-bf5cd14e9c5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:38.132316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ee6ca-526e-4a8f-921d-bf5cd14e9c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:38.133666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ee6ca-526e-4a8f-921d-bf5cd14e9c5a service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:38.156490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 744/1509] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:38.685441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de48cdb-f21d-4cd7-a3da-c220bc2ef0ee None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:38.687451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7de48cdb-f21d-4cd7-a3da-c220bc2ef0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:38.703274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de48cdb-f21d-4cd7-a3da-c220bc2ef0ee None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:38.861759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:38.863144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:38.864209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:38.864723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] PATH_INFO: `/v3/projects/1024982ef1a64b6b9230b15d7381280f` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:38.886883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1024982ef1a64b6b9230b15d7381280f)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:38.888833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING py.warnings [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:45:38.888833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: warnings.warn(msg) Aug 24 17:45:38.888833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: Aug 24 17:45:38.891297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:39.064590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1024982ef1a64b6b9230b15d7381280f', 'request_id': 'req-b0109d69-e465-4ed7-b366-a501ed2f51fc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:39.160195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1024982ef1a64b6b9230b15d7381280f', 'request_id': 'req-b0109d69-e465-4ed7-b366-a501ed2f51fc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:39.177566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1024982ef1a64b6b9230b15d7381280f', 'request_id': 'req-b0109d69-e465-4ed7-b366-a501ed2f51fc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:39.206164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] The token cache is being invalidate because project 1024982ef1a64b6b9230b15d7381280f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:45:39.206586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:39.207845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b0109d69-e465-4ed7-b366-a501ed2f51fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:45:39.209435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 766/1510] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:45:38 2022] DELETE /identity/v3/projects/1024982ef1a64b6b9230b15d7381280f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:45:48.153712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=62 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:49.203846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:05.002984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1648dc1-d8f1-47fa-9a48-235e70cd2116 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:05.003593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1648dc1-d8f1-47fa-9a48-235e70cd2116 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:05.003593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1648dc1-d8f1-47fa-9a48-235e70cd2116 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:05.005113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:47:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:05.011269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b936a-c8d3-4f3f-81db-98b5e603c167 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:05.011843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b936a-c8d3-4f3f-81db-98b5e603c167 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:05.011947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b936a-c8d3-4f3f-81db-98b5e603c167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:05.113016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-e96b936a-c8d3-4f3f-81db-98b5e603c167 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:05.114402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e96b936a-c8d3-4f3f-81db-98b5e603c167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:05.242736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:47:05.407823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1648dc1-d8f1-47fa-9a48-235e70cd2116 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:05.410175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a1648dc1-d8f1-47fa-9a48-235e70cd2116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:05.560535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1648dc1-d8f1-47fa-9a48-235e70cd2116 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:05.566792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a67a4-55c8-4c20-a2af-95dd94ee7c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:05.567421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a67a4-55c8-4c20-a2af-95dd94ee7c0f service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:05.567694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a67a4-55c8-4c20-a2af-95dd94ee7c0f service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:05.567973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a67a4-55c8-4c20-a2af-95dd94ee7c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:05.577362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a67a4-55c8-4c20-a2af-95dd94ee7c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:05.579754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a67a4-55c8-4c20-a2af-95dd94ee7c0f service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:05.592353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 746/1513] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:06.488449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8a174-8f74-458b-8f9f-6c0a2fda2a05 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:06.489132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8a174-8f74-458b-8f9f-6c0a2fda2a05 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:06.489132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8a174-8f74-458b-8f9f-6c0a2fda2a05 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:06.490414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:06.495925 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758b842-196a-4e3d-849f-caf343978ff1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:06.496399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758b842-196a-4e3d-849f-caf343978ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:06.496493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758b842-196a-4e3d-849f-caf343978ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:06.518770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-5758b842-196a-4e3d-849f-caf343978ff1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:06.520013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5758b842-196a-4e3d-849f-caf343978ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:06.531397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 747/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:06.692529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8a174-8f74-458b-8f9f-6c0a2fda2a05 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:06.708753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8a174-8f74-458b-8f9f-6c0a2fda2a05 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:06.715015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec235c73-aa12-4962-af30-086b57e0bb9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:06.715641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec235c73-aa12-4962-af30-086b57e0bb9a service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:06.716069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec235c73-aa12-4962-af30-086b57e0bb9a service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:06.716243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec235c73-aa12-4962-af30-086b57e0bb9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:06.734234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec235c73-aa12-4962-af30-086b57e0bb9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:06.738443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec235c73-aa12-4962-af30-086b57e0bb9a service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:06.747912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 769/1516] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:06.803038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedff43-b9a8-494b-81d7-c6ab2dcd6456 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:06.803604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedff43-b9a8-494b-81d7-c6ab2dcd6456 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:06.803894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedff43-b9a8-494b-81d7-c6ab2dcd6456 None None] PATH_INFO: `/v3` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:06.805394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 748/1517] 10.222.0.26 () {54 vars in 925 bytes} [Wed Aug 24 17:47:06 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:47:06.811861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b090033-061b-4c9e-90e4-2b48c634ce8e None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:06.812153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b090033-061b-4c9e-90e4-2b48c634ce8e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:06.812436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b090033-061b-4c9e-90e4-2b48c634ce8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:06.895406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-2b090033-061b-4c9e-90e4-2b48c634ce8e None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:06.896874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2b090033-061b-4c9e-90e4-2b48c634ce8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:06.918030 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:06.978508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 770/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Aug 24 17:47:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:11.077718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f023d9-4b24-48d6-b62c-cdb0a9ca4701 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:11.078306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f023d9-4b24-48d6-b62c-cdb0a9ca4701 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:11.078306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f023d9-4b24-48d6-b62c-cdb0a9ca4701 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:11.079617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 749/1519] 10.222.0.76 () {54 vars in 961 bytes} [Wed Aug 24 17:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:11.084917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d4b40-74c1-446a-904b-32e4d43b85b6 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:11.085561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d4b40-74c1-446a-904b-32e4d43b85b6 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:11.085754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d4b40-74c1-446a-904b-32e4d43b85b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:11.108843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-104d4b40-74c1-446a-904b-32e4d43b85b6 None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:11.110091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-104d4b40-74c1-446a-904b-32e4d43b85b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:11.119558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 771/1520] 10.222.0.76 () {58 vars in 1058 bytes} [Wed Aug 24 17:47:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:11.127967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f023d9-4b24-48d6-b62c-cdb0a9ca4701 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:11.144075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f023d9-4b24-48d6-b62c-cdb0a9ca4701 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:11.149427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9049563-d86d-43e3-9a08-461392ab2618 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:11.154143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9049563-d86d-43e3-9a08-461392ab2618 service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:11.154143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9049563-d86d-43e3-9a08-461392ab2618 service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:11.154143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9049563-d86d-43e3-9a08-461392ab2618 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:11.161146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9049563-d86d-43e3-9a08-461392ab2618 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:11.162821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9049563-d86d-43e3-9a08-461392ab2618 service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:11.173010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 750/1521] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:47:18.086629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3747f5-3848-41d6-aec6-4525a09c8a43 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:18.087321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3747f5-3848-41d6-aec6-4525a09c8a43 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:18.087321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3747f5-3848-41d6-aec6-4525a09c8a43 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:18.088546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:18.094413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e173b8-da2c-4132-a51e-d3f90ef55b59 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:18.094922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e173b8-da2c-4132-a51e-d3f90ef55b59 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:18.095070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e173b8-da2c-4132-a51e-d3f90ef55b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:18.115581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-93e173b8-da2c-4132-a51e-d3f90ef55b59 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:18.116797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-93e173b8-da2c-4132-a51e-d3f90ef55b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:18.125247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:18.282362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3747f5-3848-41d6-aec6-4525a09c8a43 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:18.298917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3747f5-3848-41d6-aec6-4525a09c8a43 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:18.304803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84940a33-e53e-470c-b7c7-84eb5d196078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:18.305408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84940a33-e53e-470c-b7c7-84eb5d196078 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:18.305689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84940a33-e53e-470c-b7c7-84eb5d196078 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:18.305993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84940a33-e53e-470c-b7c7-84eb5d196078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:18.315180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84940a33-e53e-470c-b7c7-84eb5d196078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:18.317144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84940a33-e53e-470c-b7c7-84eb5d196078 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:18.328225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 773/1524] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:19.170359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db179-d500-477e-82d3-93eb45d2e1b7 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:19.170970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db179-d500-477e-82d3-93eb45d2e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:19.170970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db179-d500-477e-82d3-93eb45d2e1b7 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:19.172188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:19.178059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503271-1c92-4cd1-bc57-d5ec6564ae97 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:19.178558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503271-1c92-4cd1-bc57-d5ec6564ae97 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:19.178682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503271-1c92-4cd1-bc57-d5ec6564ae97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:19.199702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-da503271-1c92-4cd1-bc57-d5ec6564ae97 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:19.200939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-da503271-1c92-4cd1-bc57-d5ec6564ae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:19.209529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:19.366643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403db179-d500-477e-82d3-93eb45d2e1b7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:19.382038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403db179-d500-477e-82d3-93eb45d2e1b7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:19.387400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4cdb6-d472-484c-9340-4cde0a9d2865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:19.388000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4cdb6-d472-484c-9340-4cde0a9d2865 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:19.388272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4cdb6-d472-484c-9340-4cde0a9d2865 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:19.388541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4cdb6-d472-484c-9340-4cde0a9d2865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:19.397444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4cdb6-d472-484c-9340-4cde0a9d2865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:19.399057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4cdb6-d472-484c-9340-4cde0a9d2865 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:19.409563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 753/1527] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:23.269083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2ab3f-faa8-410e-86a2-e4aa6abbd10b None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:23.269733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2ab3f-faa8-410e-86a2-e4aa6abbd10b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:23.269733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2ab3f-faa8-410e-86a2-e4aa6abbd10b None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:23.271144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:23.277175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304361-de89-4ebf-a357-626bab5e5fc9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:23.277941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304361-de89-4ebf-a357-626bab5e5fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:23.277941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304361-de89-4ebf-a357-626bab5e5fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:23.301603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ed304361-de89-4ebf-a357-626bab5e5fc9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:23.302904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ed304361-de89-4ebf-a357-626bab5e5fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:23.311065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:23.606817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2ab3f-faa8-410e-86a2-e4aa6abbd10b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:23.608528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-60e2ab3f-faa8-410e-86a2-e4aa6abbd10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:23.703641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2ab3f-faa8-410e-86a2-e4aa6abbd10b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:23.707636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719ba43-520f-4791-b127-7e54ee2709ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:23.708256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719ba43-520f-4791-b127-7e54ee2709ed service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:23.708536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719ba43-520f-4791-b127-7e54ee2709ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:23.708836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719ba43-520f-4791-b127-7e54ee2709ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:23.718909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719ba43-520f-4791-b127-7e54ee2709ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:23.720360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719ba43-520f-4791-b127-7e54ee2709ed service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:23.730834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 776/1530] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:25.671412 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6e442-3c73-453c-a372-00cd4bccd6c0 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:25.671984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6e442-3c73-453c-a372-00cd4bccd6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:25.671984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6e442-3c73-453c-a372-00cd4bccd6c0 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:25.673328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:25.679823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d86c0-fd2d-486f-b6a9-c17bb2a2ceae None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:25.680328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d86c0-fd2d-486f-b6a9-c17bb2a2ceae None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:25.680442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d86c0-fd2d-486f-b6a9-c17bb2a2ceae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:25.700969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-e66d86c0-fd2d-486f-b6a9-c17bb2a2ceae None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:25.702241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e66d86c0-fd2d-486f-b6a9-c17bb2a2ceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:25.711271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:26.003944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6e442-3c73-453c-a372-00cd4bccd6c0 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:26.005682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-17e6e442-3c73-453c-a372-00cd4bccd6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:26.018990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6e442-3c73-453c-a372-00cd4bccd6c0 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:26.022518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d54358-ef09-4ee6-84a5-4775c0788cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:26.023129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d54358-ef09-4ee6-84a5-4775c0788cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:26.023396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d54358-ef09-4ee6-84a5-4775c0788cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:26.023662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d54358-ef09-4ee6-84a5-4775c0788cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:26.032644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d54358-ef09-4ee6-84a5-4775c0788cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:26.034349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d54358-ef09-4ee6-84a5-4775c0788cb4 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:26.044091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 756/1533] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:29.521314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73747b6a-fd06-48f1-86ed-7e0ef986f6d9 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:29.521934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73747b6a-fd06-48f1-86ed-7e0ef986f6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:29.521934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73747b6a-fd06-48f1-86ed-7e0ef986f6d9 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:29.523341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:29.530730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffffb3-f173-4967-b094-e9357ac9b777 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:29.533729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffffb3-f173-4967-b094-e9357ac9b777 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:29.533729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffffb3-f173-4967-b094-e9357ac9b777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:29.544983 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=81 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:29.564916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b3ffffb3-f173-4967-b094-e9357ac9b777 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:29.566665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b3ffffb3-f173-4967-b094-e9357ac9b777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:29.575018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:30.160515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73747b6a-fd06-48f1-86ed-7e0ef986f6d9 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:30.162372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-73747b6a-fd06-48f1-86ed-7e0ef986f6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:30.249089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:30.269692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73747b6a-fd06-48f1-86ed-7e0ef986f6d9 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:30.275275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf306908-33c2-47b9-9490-43633a543263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:30.276318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf306908-33c2-47b9-9490-43633a543263 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:30.276820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf306908-33c2-47b9-9490-43633a543263 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:30.277316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf306908-33c2-47b9-9490-43633a543263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:30.295717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf306908-33c2-47b9-9490-43633a543263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:30.298694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf306908-33c2-47b9-9490-43633a543263 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:30.317011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 779/1536] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:32.305190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605d2e5-66d3-4d7d-aef5-e019af35638e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:32.305809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605d2e5-66d3-4d7d-aef5-e019af35638e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:32.305809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605d2e5-66d3-4d7d-aef5-e019af35638e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:32.307144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:47:32.313080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba60469-984f-42fa-a8d1-05015ec2a690 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:32.313698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba60469-984f-42fa-a8d1-05015ec2a690 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:32.313798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba60469-984f-42fa-a8d1-05015ec2a690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:32.335735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3ba60469-984f-42fa-a8d1-05015ec2a690 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:32.340761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3ba60469-984f-42fa-a8d1-05015ec2a690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:32.346024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:47:32.926031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605d2e5-66d3-4d7d-aef5-e019af35638e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:32.927876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1605d2e5-66d3-4d7d-aef5-e019af35638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:32.942377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605d2e5-66d3-4d7d-aef5-e019af35638e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:32.946151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb20dd-e4cd-4e39-8767-f52cf4fa8ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:32.946763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb20dd-e4cd-4e39-8767-f52cf4fa8ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:32.947038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb20dd-e4cd-4e39-8767-f52cf4fa8ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:32.947304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb20dd-e4cd-4e39-8767-f52cf4fa8ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:32.962847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb20dd-e4cd-4e39-8767-f52cf4fa8ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:32.965491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb20dd-e4cd-4e39-8767-f52cf4fa8ea0 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:32.981038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 759/1539] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:42.330584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=11 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:42.975772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=8 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:58.765222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122639f7-8e02-4509-b313-dec21bf55cde None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:58.765940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122639f7-8e02-4509-b313-dec21bf55cde None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:58.765940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122639f7-8e02-4509-b313-dec21bf55cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:58.795439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-122639f7-8e02-4509-b313-dec21bf55cde None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:58.796715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-122639f7-8e02-4509-b313-dec21bf55cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:58.806746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 781/1540] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:58.815767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198da9-d206-4736-aaed-ae3261b64cc9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:58.817207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198da9-d206-4736-aaed-ae3261b64cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:58.817695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198da9-d206-4736-aaed-ae3261b64cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:58.824920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5c1e5-b2fa-4828-b9d9-28be62e000fc None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:58.825211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5c1e5-b2fa-4828-b9d9-28be62e000fc None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:58.825487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5c1e5-b2fa-4828-b9d9-28be62e000fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:58.855047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f8198da9-d206-4736-aaed-ae3261b64cc9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:58.855297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-69c5c1e5-b2fa-4828-b9d9-28be62e000fc None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:58.856570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-69c5c1e5-b2fa-4828-b9d9-28be62e000fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:58.856735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f8198da9-d206-4736-aaed-ae3261b64cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:58.872407 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 760/1541] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:58.873255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 782/1542] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:58.882304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8198da9-d206-4736-aaed-ae3261b64cc9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:58.885633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7858f37-2234-47d7-bc5a-8d2dd889142c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:58.886318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7858f37-2234-47d7-bc5a-8d2dd889142c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:58.886771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7858f37-2234-47d7-bc5a-8d2dd889142c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:58.899277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8198da9-d206-4736-aaed-ae3261b64cc9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:58.916638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c8b2e-8a52-417f-b739-1ddb6157c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:58.917625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c8b2e-8a52-417f-b739-1ddb6157c729 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:58.918097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c8b2e-8a52-417f-b739-1ddb6157c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:58.918541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c8b2e-8a52-417f-b739-1ddb6157c729 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:58.919968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c8b2e-8a52-417f-b739-1ddb6157c729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:58.920607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c8b2e-8a52-417f-b739-1ddb6157c729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:58.921912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING py.warnings [None req-4a0c8b2e-8a52-417f-b739-1ddb6157c729 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:48:58.921912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: warnings.warn(msg) Aug 24 17:48:58.921912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: Aug 24 17:48:58.923216 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c8b2e-8a52-417f-b739-1ddb6157c729 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:58.933357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b7858f37-2234-47d7-bc5a-8d2dd889142c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:58.934614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b7858f37-2234-47d7-bc5a-8d2dd889142c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:58.943152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 761/1543] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:58.944883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea33832-06bb-4bb0-bcfe-87de756c3b03 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:58.945337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea33832-06bb-4bb0-bcfe-87de756c3b03 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:58.946012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea33832-06bb-4bb0-bcfe-87de756c3b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:58.959788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 783/1544] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:58.960644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7858f37-2234-47d7-bc5a-8d2dd889142c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:58.970753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7858f37-2234-47d7-bc5a-8d2dd889142c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:58.980912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-bea33832-06bb-4bb0-bcfe-87de756c3b03 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:58.982428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bea33832-06bb-4bb0-bcfe-87de756c3b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:58.990801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f709145-0169-47a9-a061-64459f919c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:58.998196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f709145-0169-47a9-a061-64459f919c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:58.998196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f709145-0169-47a9-a061-64459f919c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:58.998196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f709145-0169-47a9-a061-64459f919c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:58.998196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f709145-0169-47a9-a061-64459f919c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:58.998196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING py.warnings [None req-1f709145-0169-47a9-a061-64459f919c62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:48:58.998196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: warnings.warn(msg) Aug 24 17:48:58.998196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: Aug 24 17:48:58.998849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f709145-0169-47a9-a061-64459f919c62 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.001843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 762/1545] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.002752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea33832-06bb-4bb0-bcfe-87de756c3b03 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.021690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea33832-06bb-4bb0-bcfe-87de756c3b03 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.030713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4cc0a-e499-408e-b6a0-3ade2b63fb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.031362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4cc0a-e499-408e-b6a0-3ade2b63fb48 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.031643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4cc0a-e499-408e-b6a0-3ade2b63fb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.032173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4cc0a-e499-408e-b6a0-3ade2b63fb48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.032858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4cc0a-e499-408e-b6a0-3ade2b63fb48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:59.033166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4cc0a-e499-408e-b6a0-3ade2b63fb48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.036692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4cc0a-e499-408e-b6a0-3ade2b63fb48 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.059855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 763/1546] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:59.061169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b4a6-6065-4136-a974-39958e7643a7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.067273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b4a6-6065-4136-a974-39958e7643a7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.067273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b4a6-6065-4136-a974-39958e7643a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.099061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-8e18b4a6-6065-4136-a974-39958e7643a7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.100355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8e18b4a6-6065-4136-a974-39958e7643a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.109979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 764/1547] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.110814 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18b4a6-6065-4136-a974-39958e7643a7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.127757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18b4a6-6065-4136-a974-39958e7643a7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.138758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 784/1548] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:58 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:59.146417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44c49e-308c-40f7-9385-a8fdb8115b8d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.146791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44c49e-308c-40f7-9385-a8fdb8115b8d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.147130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44c49e-308c-40f7-9385-a8fdb8115b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.155257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8078d-e53f-4c67-8364-f41feb70d565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.155876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8078d-e53f-4c67-8364-f41feb70d565 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.156404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8078d-e53f-4c67-8364-f41feb70d565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.156681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8078d-e53f-4c67-8364-f41feb70d565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.157592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8078d-e53f-4c67-8364-f41feb70d565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.159503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8078d-e53f-4c67-8364-f41feb70d565 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.170904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-0f44c49e-308c-40f7-9385-a8fdb8115b8d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.172562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0f44c49e-308c-40f7-9385-a8fdb8115b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.250894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 765/1549] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:59.250894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8078d-e53f-4c67-8364-f41feb70d565 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.252806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 785/1550] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.253644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44c49e-308c-40f7-9385-a8fdb8115b8d None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.268738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44c49e-308c-40f7-9385-a8fdb8115b8d None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.326496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8078d-e53f-4c67-8364-f41feb70d565 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.329450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07f34c-406c-45ca-a125-8a82c9ad792d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.330095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f34c-406c-45ca-a125-8a82c9ad792d admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.330388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f34c-406c-45ca-a125-8a82c9ad792d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.330644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f34c-406c-45ca-a125-8a82c9ad792d admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.331387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07f34c-406c-45ca-a125-8a82c9ad792d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.331510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901fa73-9871-48f0-814c-1109efcebdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.332712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901fa73-9871-48f0-814c-1109efcebdd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.332933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901fa73-9871-48f0-814c-1109efcebdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.333338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901fa73-9871-48f0-814c-1109efcebdd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.333718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07f34c-406c-45ca-a125-8a82c9ad792d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.334046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901fa73-9871-48f0-814c-1109efcebdd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:59.334352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901fa73-9871-48f0-814c-1109efcebdd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.335984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901fa73-9871-48f0-814c-1109efcebdd2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.350471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 766/1551] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:59.354613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0adbf5-7b3d-4b23-be13-0604b24efafb None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.354613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0adbf5-7b3d-4b23-be13-0604b24efafb None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.354613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0adbf5-7b3d-4b23-be13-0604b24efafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.374266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ca0adbf5-7b3d-4b23-be13-0604b24efafb None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.377550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ca0adbf5-7b3d-4b23-be13-0604b24efafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.388660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 767/1552] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.390145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145977e-7baa-4ea1-9d20-425400a14ed7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.390434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145977e-7baa-4ea1-9d20-425400a14ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.390745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145977e-7baa-4ea1-9d20-425400a14ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.402595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 786/1553] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:59.402595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f2598-2e10-40e7-84b2-cd8d3dab703f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.402595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f2598-2e10-40e7-84b2-cd8d3dab703f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.406421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f2598-2e10-40e7-84b2-cd8d3dab703f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.416907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e145977e-7baa-4ea1-9d20-425400a14ed7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.418204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e145977e-7baa-4ea1-9d20-425400a14ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.427476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 768/1554] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.428325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145977e-7baa-4ea1-9d20-425400a14ed7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.434737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-797f2598-2e10-40e7-84b2-cd8d3dab703f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.435969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-797f2598-2e10-40e7-84b2-cd8d3dab703f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.443364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145977e-7baa-4ea1-9d20-425400a14ed7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.446821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 787/1555] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.446821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea088ae-701e-4369-a268-fe90c492d7ec None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.446821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea088ae-701e-4369-a268-fe90c492d7ec None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.447162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea088ae-701e-4369-a268-fe90c492d7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.450860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba889-3523-41a1-9d96-22580aff37bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.451489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba889-3523-41a1-9d96-22580aff37bb admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.451773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba889-3523-41a1-9d96-22580aff37bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.452147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba889-3523-41a1-9d96-22580aff37bb admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.452800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ba889-3523-41a1-9d96-22580aff37bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.453838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING py.warnings [None req-4d3ba889-3523-41a1-9d96-22580aff37bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:48:59.453838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: warnings.warn(msg) Aug 24 17:48:59.453838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: Aug 24 17:48:59.455202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ba889-3523-41a1-9d96-22580aff37bb admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.475214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3ea088ae-701e-4369-a268-fe90c492d7ec None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.476644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3ea088ae-701e-4369-a268-fe90c492d7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.485265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 788/1556] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.486712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963d19b-6cbc-4e95-8b6e-eda46f6351f9 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.486994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963d19b-6cbc-4e95-8b6e-eda46f6351f9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.487270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963d19b-6cbc-4e95-8b6e-eda46f6351f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.508677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-5963d19b-6cbc-4e95-8b6e-eda46f6351f9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.510098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5963d19b-6cbc-4e95-8b6e-eda46f6351f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.519163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 789/1557] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.524205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963d19b-6cbc-4e95-8b6e-eda46f6351f9 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.535770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963d19b-6cbc-4e95-8b6e-eda46f6351f9 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.539148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 769/1558] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:59.540478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fe94e-07e6-41cb-b37e-73dc271c271a None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.540767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fe94e-07e6-41cb-b37e-73dc271c271a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.541043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fe94e-07e6-41cb-b37e-73dc271c271a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.545998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6da60-8990-4fa7-a790-d77bcb56ec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.546702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6da60-8990-4fa7-a790-d77bcb56ec14 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.547044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6da60-8990-4fa7-a790-d77bcb56ec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.547350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6da60-8990-4fa7-a790-d77bcb56ec14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.547991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6da60-8990-4fa7-a790-d77bcb56ec14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.550162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6da60-8990-4fa7-a790-d77bcb56ec14 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.563707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-1e8fe94e-07e6-41cb-b37e-73dc271c271a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.564947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1e8fe94e-07e6-41cb-b37e-73dc271c271a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.573522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 770/1559] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.574450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fe94e-07e6-41cb-b37e-73dc271c271a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.584459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fe94e-07e6-41cb-b37e-73dc271c271a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.589182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60299ca7-54bd-49fa-8003-52efb53de2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.589775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60299ca7-54bd-49fa-8003-52efb53de2fd admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.590092 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60299ca7-54bd-49fa-8003-52efb53de2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.590368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60299ca7-54bd-49fa-8003-52efb53de2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.591014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60299ca7-54bd-49fa-8003-52efb53de2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.592650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60299ca7-54bd-49fa-8003-52efb53de2fd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.603303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 771/1560] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:59.604594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5c67e-5eb2-483b-b0bc-759f9edb63b7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.604880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5c67e-5eb2-483b-b0bc-759f9edb63b7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.605149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5c67e-5eb2-483b-b0bc-759f9edb63b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.625642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b8e5c67e-5eb2-483b-b0bc-759f9edb63b7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.627272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b8e5c67e-5eb2-483b-b0bc-759f9edb63b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.635886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 772/1561] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.637132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea866e-f329-40bf-aeff-b3cb396b059d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.637420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea866e-f329-40bf-aeff-b3cb396b059d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.637684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea866e-f329-40bf-aeff-b3cb396b059d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.657181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e1ea866e-f329-40bf-aeff-b3cb396b059d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.658431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e1ea866e-f329-40bf-aeff-b3cb396b059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.667106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 773/1562] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.667902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea866e-f329-40bf-aeff-b3cb396b059d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.678494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea866e-f329-40bf-aeff-b3cb396b059d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.683089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864be646-53ba-45a2-96be-9f393d58b332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.683700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864be646-53ba-45a2-96be-9f393d58b332 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.683982 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864be646-53ba-45a2-96be-9f393d58b332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.684252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864be646-53ba-45a2-96be-9f393d58b332 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.684906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864be646-53ba-45a2-96be-9f393d58b332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:59.685212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864be646-53ba-45a2-96be-9f393d58b332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.686848 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864be646-53ba-45a2-96be-9f393d58b332 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.700736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 774/1563] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:59.700736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5eb1e-b992-489a-9b1e-7f7cd2bf29a4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.711420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5eb1e-b992-489a-9b1e-7f7cd2bf29a4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.711420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5eb1e-b992-489a-9b1e-7f7cd2bf29a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.721440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-79e5eb1e-b992-489a-9b1e-7f7cd2bf29a4 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:59.723438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-79e5eb1e-b992-489a-9b1e-7f7cd2bf29a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.725772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:59.731719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 775/1564] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:59.732527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5eb1e-b992-489a-9b1e-7f7cd2bf29a4 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.737241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 790/1565] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:59.738791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6da60-8990-4fa7-a790-d77bcb56ec14 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.740811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:59.744914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5eb1e-b992-489a-9b1e-7f7cd2bf29a4 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.803400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b60a89-6cdc-430a-854e-041261345cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.803918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b60a89-6cdc-430a-854e-041261345cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.804205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b60a89-6cdc-430a-854e-041261345cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.804515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b60a89-6cdc-430a-854e-041261345cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.805116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b60a89-6cdc-430a-854e-041261345cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.806221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6da60-8990-4fa7-a790-d77bcb56ec14 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.806745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b60a89-6cdc-430a-854e-041261345cd0 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.812861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.813871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.814319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.814644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.815976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:59.816595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.818549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.821488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 776/1566] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:59.822953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b60a89-6cdc-430a-854e-041261345cd0 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.833571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b60a89-6cdc-430a-854e-041261345cd0 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.834452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 791/1567] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:59.835705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.838513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242fd10-5792-4f6a-a6de-5e97c29ccb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.839121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242fd10-5792-4f6a-a6de-5e97c29ccb46 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.839423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242fd10-5792-4f6a-a6de-5e97c29ccb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.839697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242fd10-5792-4f6a-a6de-5e97c29ccb46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.840339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242fd10-5792-4f6a-a6de-5e97c29ccb46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.841979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242fd10-5792-4f6a-a6de-5e97c29ccb46 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.846751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58600f00-94d0-4185-924e-8cfcfb2a6222 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.852501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.853133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.853361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.853635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.854116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 777/1568] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:59.854324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:59.854628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.855298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242fd10-5792-4f6a-a6de-5e97c29ccb46 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.856297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.865391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242fd10-5792-4f6a-a6de-5e97c29ccb46 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.870063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.870651 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.870786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 792/1569] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:59.870933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.871206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.871595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.871848 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:59.872207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.873800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.882398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d5759-ffd6-4894-bada-b61ea0952967 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.887311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdfc20-4353-4280-a4bf-163b6f58977b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.887912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdfc20-4353-4280-a4bf-163b6f58977b admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.888197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdfc20-4353-4280-a4bf-163b6f58977b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.888495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdfc20-4353-4280-a4bf-163b6f58977b admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.888674 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 778/1570] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:59.889424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdfc20-4353-4280-a4bf-163b6f58977b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.889556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.891388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdfc20-4353-4280-a4bf-163b6f58977b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.903449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd98f2-4dd7-454c-b436-5ffd4d508f18 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.909742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.911034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.911311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.911649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.912228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:59.912528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.914220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.926763 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 779/1571] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:59.927570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.938086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb3b16-f9bf-4323-9977-41cd17cb07f7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.943022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faf40c-2467-49d3-92a4-3567560e124e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.943639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faf40c-2467-49d3-92a4-3567560e124e admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.946177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faf40c-2467-49d3-92a4-3567560e124e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.946177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faf40c-2467-49d3-92a4-3567560e124e admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.946177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faf40c-2467-49d3-92a4-3567560e124e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.946849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faf40c-2467-49d3-92a4-3567560e124e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.039771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 793/1572] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/projects => generated 405 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:00.039771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdfc20-4353-4280-a4bf-163b6f58977b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.101383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdfc20-4353-4280-a4bf-163b6f58977b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.103521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 780/1573] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:59 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:00.104749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faf40c-2467-49d3-92a4-3567560e124e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.108845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a57c4-943a-42d2-a442-c729827098ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.108845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a57c4-943a-42d2-a442-c729827098ce admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.108845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a57c4-943a-42d2-a442-c729827098ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.108845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a57c4-943a-42d2-a442-c729827098ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.109339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a57c4-943a-42d2-a442-c729827098ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.110066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a57c4-943a-42d2-a442-c729827098ce admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.115580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faf40c-2467-49d3-92a4-3567560e124e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.120291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1e7e7-a938-4bb4-ad5a-7674710f343e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.124155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 794/1574] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:00.124290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1e7e7-a938-4bb4-ad5a-7674710f343e admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.124290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1e7e7-a938-4bb4-ad5a-7674710f343e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.124290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1e7e7-a938-4bb4-ad5a-7674710f343e admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.124580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a57c4-943a-42d2-a442-c729827098ce admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.124680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1e7e7-a938-4bb4-ad5a-7674710f343e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.124680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1e7e7-a938-4bb4-ad5a-7674710f343e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.135626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a57c4-943a-42d2-a442-c729827098ce admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.142042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.143302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.143736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.144021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] PATH_INFO: `/v3/projects/0a4d7fadea104c569ca29ff9be48089a/users/251bc85f89774142a97acd77895b14e9/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.161051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4d7fadea104c569ca29ff9be48089a, user_id=251bc85f89774142a97acd77895b14e9, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.162185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING py.warnings [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:49:00.162185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: warnings.warn(msg) Aug 24 17:49:00.162185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: Aug 24 17:49:00.164169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.207901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 781/1575] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:00.208846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1e7e7-a938-4bb4-ad5a-7674710f343e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.228513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 795/1576] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:00 2022] PUT /identity/v3/projects/0a4d7fadea104c569ca29ff9be48089a/users/251bc85f89774142a97acd77895b14e9/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:00.229343 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.267884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:00.277825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1e7e7-a938-4bb4-ad5a-7674710f343e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.301877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78041371-d2a1-44b4-b71a-fc30fc213c47 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.306634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26df4f-1998-4e94-bff8-36bf12704c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.307240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26df4f-1998-4e94-bff8-36bf12704c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.307525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26df4f-1998-4e94-bff8-36bf12704c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.307802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26df4f-1998-4e94-bff8-36bf12704c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.308438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26df4f-1998-4e94-bff8-36bf12704c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.308581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:00.310388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26df4f-1998-4e94-bff8-36bf12704c59 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.340295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aca4d1-5426-4a69-991d-e932b978fe5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.340918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aca4d1-5426-4a69-991d-e932b978fe5d admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.341404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aca4d1-5426-4a69-991d-e932b978fe5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.341729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aca4d1-5426-4a69-991d-e932b978fe5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.342617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aca4d1-5426-4a69-991d-e932b978fe5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.345363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aca4d1-5426-4a69-991d-e932b978fe5d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.396251 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 796/1577] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:00.397134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26df4f-1998-4e94-bff8-36bf12704c59 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.418627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 782/1578] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:00.422906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aca4d1-5426-4a69-991d-e932b978fe5d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.471043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26df4f-1998-4e94-bff8-36bf12704c59 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.492654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aca4d1-5426-4a69-991d-e932b978fe5d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.497646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc92e4-cdb7-48c0-a9a2-6214e9d8c50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.498307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc92e4-cdb7-48c0-a9a2-6214e9d8c50b admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.498594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc92e4-cdb7-48c0-a9a2-6214e9d8c50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.499012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc92e4-cdb7-48c0-a9a2-6214e9d8c50b admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.499662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc92e4-cdb7-48c0-a9a2-6214e9d8c50b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.501703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc92e4-cdb7-48c0-a9a2-6214e9d8c50b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.534061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfdc08-cb40-4d2a-828d-fcfa2df5546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.534673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfdc08-cb40-4d2a-828d-fcfa2df5546a admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.535017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfdc08-cb40-4d2a-828d-fcfa2df5546a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.535272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfdc08-cb40-4d2a-828d-fcfa2df5546a admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.536073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfdc08-cb40-4d2a-828d-fcfa2df5546a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.538018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfdc08-cb40-4d2a-828d-fcfa2df5546a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.594257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 783/1579] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/users => generated 303 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:00.595124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc92e4-cdb7-48c0-a9a2-6214e9d8c50b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.605283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc92e4-cdb7-48c0-a9a2-6214e9d8c50b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.610041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c900931-02ca-44d5-83e7-c55bc37f8a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.610685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900931-02ca-44d5-83e7-c55bc37f8a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.610930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900931-02ca-44d5-83e7-c55bc37f8a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.611206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900931-02ca-44d5-83e7-c55bc37f8a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.611860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c900931-02ca-44d5-83e7-c55bc37f8a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.614072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c900931-02ca-44d5-83e7-c55bc37f8a5d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.624339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 797/1580] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:00.627955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 784/1581] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:00.628136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfdc08-cb40-4d2a-828d-fcfa2df5546a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.628248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c900931-02ca-44d5-83e7-c55bc37f8a5d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.638071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c900931-02ca-44d5-83e7-c55bc37f8a5d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.683892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:00.699699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfdc08-cb40-4d2a-828d-fcfa2df5546a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.704779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebd651-46a8-4b4a-9c11-8ca50cf5ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.705500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebd651-46a8-4b4a-9c11-8ca50cf5ee79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.705703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccd64e-9976-4c20-a225-bd49316fc38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.705877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebd651-46a8-4b4a-9c11-8ca50cf5ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.706173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebd651-46a8-4b4a-9c11-8ca50cf5ee79 admin admin] PATH_INFO: `/v3/projects/ca875d9739724041937d23d7ca212d91/users/d9aa128932f94a57a7b0bdf3f18ed1bf/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.706349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd64e-9976-4c20-a225-bd49316fc38b admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.706631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd64e-9976-4c20-a225-bd49316fc38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.706940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd64e-9976-4c20-a225-bd49316fc38b admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.707859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccd64e-9976-4c20-a225-bd49316fc38b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.710387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccd64e-9976-4c20-a225-bd49316fc38b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.723700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebd651-46a8-4b4a-9c11-8ca50cf5ee79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca875d9739724041937d23d7ca212d91, user_id=d9aa128932f94a57a7b0bdf3f18ed1bf, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.728478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebd651-46a8-4b4a-9c11-8ca50cf5ee79 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.817827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 785/1582] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:00.819185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3b550-f74f-40be-aece-614540a4816c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.819480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3b550-f74f-40be-aece-614540a4816c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.819756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3b550-f74f-40be-aece-614540a4816c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.828706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 798/1583] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:00 2022] PUT /identity/v3/projects/ca875d9739724041937d23d7ca212d91/users/d9aa128932f94a57a7b0bdf3f18ed1bf/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:00.829517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebd651-46a8-4b4a-9c11-8ca50cf5ee79 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.848649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2bf3b550-f74f-40be-aece-614540a4816c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:00.850055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2bf3b550-f74f-40be-aece-614540a4816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:00.869539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:00.907835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebd651-46a8-4b4a-9c11-8ca50cf5ee79 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.913093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b247f7-1064-419d-8f90-85c4ddc4caad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.913770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b247f7-1064-419d-8f90-85c4ddc4caad admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.914137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b247f7-1064-419d-8f90-85c4ddc4caad admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.914477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b247f7-1064-419d-8f90-85c4ddc4caad admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.915331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b247f7-1064-419d-8f90-85c4ddc4caad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.917265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b247f7-1064-419d-8f90-85c4ddc4caad admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:00.919952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 786/1584] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:00.921084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3b550-f74f-40be-aece-614540a4816c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:00.932406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3b550-f74f-40be-aece-614540a4816c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:00.937193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4465a1-cca4-41b8-b49c-2a5226d85b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:00.938003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4465a1-cca4-41b8-b49c-2a5226d85b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:00.938141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4465a1-cca4-41b8-b49c-2a5226d85b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:00.938352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4465a1-cca4-41b8-b49c-2a5226d85b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:00.938989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4465a1-cca4-41b8-b49c-2a5226d85b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:00.940970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4465a1-cca4-41b8-b49c-2a5226d85b5e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.024552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 799/1585] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:01.025666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b247f7-1064-419d-8f90-85c4ddc4caad admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.037187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b247f7-1064-419d-8f90-85c4ddc4caad admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.042311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d82f9-f6c7-42bc-abd0-d306b84b138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.043116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d82f9-f6c7-42bc-abd0-d306b84b138c admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.043116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d82f9-f6c7-42bc-abd0-d306b84b138c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.043827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d82f9-f6c7-42bc-abd0-d306b84b138c admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.044545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d82f9-f6c7-42bc-abd0-d306b84b138c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.046190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d82f9-f6c7-42bc-abd0-d306b84b138c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.052457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 787/1586] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:00 2022] POST /identity/v3/users => generated 286 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:01.053285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4465a1-cca4-41b8-b49c-2a5226d85b5e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.064433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 800/1587] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.064433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d82f9-f6c7-42bc-abd0-d306b84b138c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.064709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4465a1-cca4-41b8-b49c-2a5226d85b5e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.068336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226feae-f0d5-46ba-a90a-b268b360473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.068938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226feae-f0d5-46ba-a90a-b268b360473d admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.069218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226feae-f0d5-46ba-a90a-b268b360473d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.069518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226feae-f0d5-46ba-a90a-b268b360473d admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.070326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226feae-f0d5-46ba-a90a-b268b360473d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.072327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226feae-f0d5-46ba-a90a-b268b360473d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.073957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d82f9-f6c7-42bc-abd0-d306b84b138c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.079391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8aa4-b20a-46e2-8885-62e1b1399b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.080397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8aa4-b20a-46e2-8885-62e1b1399b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.083441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8aa4-b20a-46e2-8885-62e1b1399b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.083441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8aa4-b20a-46e2-8885-62e1b1399b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.083441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f8aa4-b20a-46e2-8885-62e1b1399b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.084027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f8aa4-b20a-46e2-8885-62e1b1399b4b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.095537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 801/1588] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.096453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8aa4-b20a-46e2-8885-62e1b1399b4b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.106836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8aa4-b20a-46e2-8885-62e1b1399b4b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.111463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34906e3a-4f9a-41a1-be43-461fe090728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.112075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34906e3a-4f9a-41a1-be43-461fe090728b admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.112354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34906e3a-4f9a-41a1-be43-461fe090728b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.112630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34906e3a-4f9a-41a1-be43-461fe090728b admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.113931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34906e3a-4f9a-41a1-be43-461fe090728b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.115512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34906e3a-4f9a-41a1-be43-461fe090728b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.126644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 802/1589] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.130954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83fd45-ed74-4fcf-a0f1-6b43dc64cfad None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.130954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83fd45-ed74-4fcf-a0f1-6b43dc64cfad None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.130954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83fd45-ed74-4fcf-a0f1-6b43dc64cfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.157054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4e83fd45-ed74-4fcf-a0f1-6b43dc64cfad None None] MFA Rules not processed for user `d9aa128932f94a57a7b0bdf3f18ed1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:01.158332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4e83fd45-ed74-4fcf-a0f1-6b43dc64cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:01.220695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 788/1590] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:01 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:01.220695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226feae-f0d5-46ba-a90a-b268b360473d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.229580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 803/1591] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:01.229580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83fd45-ed74-4fcf-a0f1-6b43dc64cfad None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.237438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83fd45-ed74-4fcf-a0f1-6b43dc64cfad None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.243875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf7d8-c275-4ea9-b86a-65aab7f3434c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.244505 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf7d8-c275-4ea9-b86a-65aab7f3434c admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.244786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf7d8-c275-4ea9-b86a-65aab7f3434c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.245057 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf7d8-c275-4ea9-b86a-65aab7f3434c admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.245806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edf7d8-c275-4ea9-b86a-65aab7f3434c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.247587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edf7d8-c275-4ea9-b86a-65aab7f3434c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.259349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 804/1592] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.259349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf7d8-c275-4ea9-b86a-65aab7f3434c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.266577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:01.271750 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf7d8-c275-4ea9-b86a-65aab7f3434c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.276629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c33a1-6fe3-4165-b929-5f75822a3290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.277357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c33a1-6fe3-4165-b929-5f75822a3290 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.277746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c33a1-6fe3-4165-b929-5f75822a3290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.278062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c33a1-6fe3-4165-b929-5f75822a3290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.278719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c33a1-6fe3-4165-b929-5f75822a3290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.280336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c33a1-6fe3-4165-b929-5f75822a3290 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.287322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226feae-f0d5-46ba-a90a-b268b360473d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.292205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 805/1593] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.293218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c33a1-6fe3-4165-b929-5f75822a3290 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.297822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8fb18-501a-46b9-bb96-6f05e4e0369a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.298480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8fb18-501a-46b9-bb96-6f05e4e0369a service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.298759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8fb18-501a-46b9-bb96-6f05e4e0369a service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.299029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8fb18-501a-46b9-bb96-6f05e4e0369a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.305086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c33a1-6fe3-4165-b929-5f75822a3290 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.309628 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8fb18-501a-46b9-bb96-6f05e4e0369a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.309910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f22459-8e79-49a8-a2ed-a624ec093a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.310518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22459-8e79-49a8-a2ed-a624ec093a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.310879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22459-8e79-49a8-a2ed-a624ec093a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.311108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8fb18-501a-46b9-bb96-6f05e4e0369a service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.311218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22459-8e79-49a8-a2ed-a624ec093a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.311787 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f22459-8e79-49a8-a2ed-a624ec093a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.313604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f22459-8e79-49a8-a2ed-a624ec093a6c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.321330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 789/1594] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:01.322144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8fb18-501a-46b9-bb96-6f05e4e0369a service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.324492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 806/1595] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.325406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f22459-8e79-49a8-a2ed-a624ec093a6c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.334014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8fb18-501a-46b9-bb96-6f05e4e0369a service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.337867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f22459-8e79-49a8-a2ed-a624ec093a6c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.339332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4590-cb91-458a-ae92-30a9944f0445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.340085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4590-cb91-458a-ae92-30a9944f0445 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.340377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4590-cb91-458a-ae92-30a9944f0445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.340661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4590-cb91-458a-ae92-30a9944f0445 admin admin] PATH_INFO: `/v3/projects/b8a384a5f96e488fb3c4abd75fd876fb/users/f5fcd46b400942a99dafdba6327ba49e/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.343871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8766-f27f-4d0b-9493-cb387166c33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.344600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8766-f27f-4d0b-9493-cb387166c33a admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.344955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8766-f27f-4d0b-9493-cb387166c33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.345439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8766-f27f-4d0b-9493-cb387166c33a admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.346180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c8766-f27f-4d0b-9493-cb387166c33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.349000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c8766-f27f-4d0b-9493-cb387166c33a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.362721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4590-cb91-458a-ae92-30a9944f0445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a384a5f96e488fb3c4abd75fd876fb, user_id=f5fcd46b400942a99dafdba6327ba49e, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.362914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 807/1596] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.362914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8766-f27f-4d0b-9493-cb387166c33a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.363582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4590-cb91-458a-ae92-30a9944f0445 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.373810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8766-f27f-4d0b-9493-cb387166c33a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.378649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d47166-0718-4740-ac47-0f2c49e544d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.379523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d47166-0718-4740-ac47-0f2c49e544d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.379921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d47166-0718-4740-ac47-0f2c49e544d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.380195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d47166-0718-4740-ac47-0f2c49e544d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.380995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d47166-0718-4740-ac47-0f2c49e544d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.382587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d47166-0718-4740-ac47-0f2c49e544d5 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.394755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 808/1597] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.395716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d47166-0718-4740-ac47-0f2c49e544d5 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.423676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 790/1598] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:01 2022] PUT /identity/v3/projects/b8a384a5f96e488fb3c4abd75fd876fb/users/f5fcd46b400942a99dafdba6327ba49e/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:01.424964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4590-cb91-458a-ae92-30a9944f0445 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.485162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d47166-0718-4740-ac47-0f2c49e544d5 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.508303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4590-cb91-458a-ae92-30a9944f0445 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.513154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fc5a2-8460-49e4-8932-e64a361e77e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.513784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fc5a2-8460-49e4-8932-e64a361e77e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.514104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fc5a2-8460-49e4-8932-e64a361e77e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.514375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fc5a2-8460-49e4-8932-e64a361e77e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.515149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fc5a2-8460-49e4-8932-e64a361e77e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.516694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fc5a2-8460-49e4-8932-e64a361e77e2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.533692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 791/1599] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.534643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fc5a2-8460-49e4-8932-e64a361e77e2 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.543225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:01.547437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fc5a2-8460-49e4-8932-e64a361e77e2 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.553192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fb1b7-51de-419e-9706-66572c7579a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.553811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fb1b7-51de-419e-9706-66572c7579a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.554120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fb1b7-51de-419e-9706-66572c7579a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.554254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8126-51ac-4796-bd7e-865ca12a4b24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.555428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fb1b7-51de-419e-9706-66572c7579a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.555530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8126-51ac-4796-bd7e-865ca12a4b24 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.555635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fb1b7-51de-419e-9706-66572c7579a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.555726 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8126-51ac-4796-bd7e-865ca12a4b24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.555726 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8126-51ac-4796-bd7e-865ca12a4b24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.556714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fb1b7-51de-419e-9706-66572c7579a9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.568917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 792/1600] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.572196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fb1b7-51de-419e-9706-66572c7579a9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.585110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fb1b7-51de-419e-9706-66572c7579a9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.595788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc4fbb-f951-45bc-9582-70f6782f832d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.597054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc4fbb-f951-45bc-9582-70f6782f832d admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.598317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc4fbb-f951-45bc-9582-70f6782f832d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.599330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc4fbb-f951-45bc-9582-70f6782f832d admin admin] PATH_INFO: `/v3/projects/4a6a1625930e40f4834c0fbf69b04cce/users/09774494757f4c56a077e58c1a09eacd/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.628538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc4fbb-f951-45bc-9582-70f6782f832d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6a1625930e40f4834c0fbf69b04cce, user_id=09774494757f4c56a077e58c1a09eacd, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.645240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc4fbb-f951-45bc-9582-70f6782f832d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.647851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c8126-51ac-4796-bd7e-865ca12a4b24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.649397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c8126-51ac-4796-bd7e-865ca12a4b24 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.659451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 809/1601] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:01.660748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8126-51ac-4796-bd7e-865ca12a4b24 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.672480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8126-51ac-4796-bd7e-865ca12a4b24 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.677601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254a219-d5db-4f85-80f6-210dd7ff3dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.678308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254a219-d5db-4f85-80f6-210dd7ff3dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.678832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254a219-d5db-4f85-80f6-210dd7ff3dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.679008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254a219-d5db-4f85-80f6-210dd7ff3dc0 admin admin] PATH_INFO: `/v3/projects/9398272c7a3f4a50a38df4b5dec9ac90/users/612dfebb8ebc401bb81d3d3333577566/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.705146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254a219-d5db-4f85-80f6-210dd7ff3dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9398272c7a3f4a50a38df4b5dec9ac90, user_id=612dfebb8ebc401bb81d3d3333577566, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.708889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254a219-d5db-4f85-80f6-210dd7ff3dc0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.725157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 793/1602] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:01 2022] PUT /identity/v3/projects/4a6a1625930e40f4834c0fbf69b04cce/users/09774494757f4c56a077e58c1a09eacd/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:01.727528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc4fbb-f951-45bc-9582-70f6782f832d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.789302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 810/1603] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:01 2022] PUT /identity/v3/projects/9398272c7a3f4a50a38df4b5dec9ac90/users/612dfebb8ebc401bb81d3d3333577566/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:01.791449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc947e7-e5f6-4bc9-be58-647fb93bebf7 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.791921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc947e7-e5f6-4bc9-be58-647fb93bebf7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.793329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc947e7-e5f6-4bc9-be58-647fb93bebf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.811324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:01.815065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc4fbb-f951-45bc-9582-70f6782f832d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.820073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-dfc947e7-e5f6-4bc9-be58-647fb93bebf7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:01.821376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-dfc947e7-e5f6-4bc9-be58-647fb93bebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:01.895452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1275a86-a522-4fd7-bf64-31f3f532e410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.896754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1275a86-a522-4fd7-bf64-31f3f532e410 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.896853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1275a86-a522-4fd7-bf64-31f3f532e410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.897083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1275a86-a522-4fd7-bf64-31f3f532e410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.897895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1275a86-a522-4fd7-bf64-31f3f532e410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.899759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1275a86-a522-4fd7-bf64-31f3f532e410 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.900549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 811/1604] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:01.902275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc947e7-e5f6-4bc9-be58-647fb93bebf7 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.929844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc947e7-e5f6-4bc9-be58-647fb93bebf7 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.937562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 794/1605] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:01.937562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1275a86-a522-4fd7-bf64-31f3f532e410 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:01.942784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a8af2-a35a-4fae-8b9e-b24ffc137bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.944744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a8af2-a35a-4fae-8b9e-b24ffc137bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.945798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a8af2-a35a-4fae-8b9e-b24ffc137bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.946282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a8af2-a35a-4fae-8b9e-b24ffc137bd3 admin admin] PATH_INFO: `/v3/projects/49fb7ae63dac46a3883ef566a3763870/users/181725023fc14d8e9d077d6012eb2e22/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.948531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1275a86-a522-4fd7-bf64-31f3f532e410 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:01.954549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ad297-3ef9-41bc-8a3e-5d4b8ce506dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:01.955303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ad297-3ef9-41bc-8a3e-5d4b8ce506dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:01.955581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ad297-3ef9-41bc-8a3e-5d4b8ce506dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:01.955853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ad297-3ef9-41bc-8a3e-5d4b8ce506dd admin admin] PATH_INFO: `/v3/projects/de73a596283345f2b6b87ce076767b78/users/e8e82fcb6d7549b1b3a62e346e661945/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:01.977868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a8af2-a35a-4fae-8b9e-b24ffc137bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49fb7ae63dac46a3883ef566a3763870, user_id=181725023fc14d8e9d077d6012eb2e22, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.979973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ad297-3ef9-41bc-8a3e-5d4b8ce506dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=de73a596283345f2b6b87ce076767b78, user_id=e8e82fcb6d7549b1b3a62e346e661945, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:01.982340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a8af2-a35a-4fae-8b9e-b24ffc137bd3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:01.983228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ad297-3ef9-41bc-8a3e-5d4b8ce506dd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:02.091774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 812/1606] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:01 2022] PUT /identity/v3/projects/49fb7ae63dac46a3883ef566a3763870/users/181725023fc14d8e9d077d6012eb2e22/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:02.093219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd31c-e192-43b4-bce5-439d4f3a1a0f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.093513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd31c-e192-43b4-bce5-439d4f3a1a0f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.093787 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd31c-e192-43b4-bce5-439d4f3a1a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.114652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 795/1607] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:01 2022] PUT /identity/v3/projects/de73a596283345f2b6b87ce076767b78/users/e8e82fcb6d7549b1b3a62e346e661945/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:02.116114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504ee84-768b-4d46-9607-5edd1a7c5df9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.116526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504ee84-768b-4d46-9607-5edd1a7c5df9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.116801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504ee84-768b-4d46-9607-5edd1a7c5df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.145756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-520fd31c-e192-43b4-bce5-439d4f3a1a0f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:02.147126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-520fd31c-e192-43b4-bce5-439d4f3a1a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:02.161294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0504ee84-768b-4d46-9607-5edd1a7c5df9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:02.162582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0504ee84-768b-4d46-9607-5edd1a7c5df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:02.242540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:02.247641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 813/1608] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:02.252657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fd31c-e192-43b4-bce5-439d4f3a1a0f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:02.278390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 796/1609] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:02.279989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504ee84-768b-4d46-9607-5edd1a7c5df9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:02.304607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504ee84-768b-4d46-9607-5edd1a7c5df9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:02.310695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6e1ad-4ff8-49f9-b2fa-04cba8d4e12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:02.312715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e1ad-4ff8-49f9-b2fa-04cba8d4e12b admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.312715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e1ad-4ff8-49f9-b2fa-04cba8d4e12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.312715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e1ad-4ff8-49f9-b2fa-04cba8d4e12b admin admin] PATH_INFO: `/v3/projects/40256802535d4aedb2541246a687ec76/users/2b6b0d0cbdb54a46a7e9d6936c47cc4c/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.330799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6e1ad-4ff8-49f9-b2fa-04cba8d4e12b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40256802535d4aedb2541246a687ec76, user_id=2b6b0d0cbdb54a46a7e9d6936c47cc4c, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:02.333784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6e1ad-4ff8-49f9-b2fa-04cba8d4e12b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:02.337509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fd31c-e192-43b4-bce5-439d4f3a1a0f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:02.341128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab254-7e9f-4ed8-9492-98881fb82fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:02.341733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab254-7e9f-4ed8-9492-98881fb82fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.342045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab254-7e9f-4ed8-9492-98881fb82fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.342316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab254-7e9f-4ed8-9492-98881fb82fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.355052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ab254-7e9f-4ed8-9492-98881fb82fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:02.355052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ab254-7e9f-4ed8-9492-98881fb82fa9 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:02.364995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 814/1610] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:02.366556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dac22-2110-4c59-babe-40ec6bf0882b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.366861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dac22-2110-4c59-babe-40ec6bf0882b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.367140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dac22-2110-4c59-babe-40ec6bf0882b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.398899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-171dac22-2110-4c59-babe-40ec6bf0882b None None] MFA Rules not processed for user `181725023fc14d8e9d077d6012eb2e22`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:02.400230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-171dac22-2110-4c59-babe-40ec6bf0882b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:02.424587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:02.445842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 797/1611] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:02 2022] PUT /identity/v3/projects/40256802535d4aedb2541246a687ec76/users/2b6b0d0cbdb54a46a7e9d6936c47cc4c/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:02.452853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d270e-9ad7-42bd-a116-850d8cdd7e72 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.453146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d270e-9ad7-42bd-a116-850d8cdd7e72 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.453430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d270e-9ad7-42bd-a116-850d8cdd7e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.506144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 815/1612] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:02.507317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dac22-2110-4c59-babe-40ec6bf0882b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:02.532735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dac22-2110-4c59-babe-40ec6bf0882b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:02.538343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a55d270e-9ad7-42bd-a116-850d8cdd7e72 None None] MFA Rules not processed for user `e8e82fcb6d7549b1b3a62e346e661945`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:02.542967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a55d270e-9ad7-42bd-a116-850d8cdd7e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:02.663250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f1490-fca5-4063-8780-40865e975579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:02.664960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f1490-fca5-4063-8780-40865e975579 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.665407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f1490-fca5-4063-8780-40865e975579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.665577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f1490-fca5-4063-8780-40865e975579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.671693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 798/1613] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:02.674004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d270e-9ad7-42bd-a116-850d8cdd7e72 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:02.695290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d270e-9ad7-42bd-a116-850d8cdd7e72 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:02.761356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da151f-773e-4e2e-ba5e-7ee69ec461d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:02.769188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da151f-773e-4e2e-ba5e-7ee69ec461d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.769188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da151f-773e-4e2e-ba5e-7ee69ec461d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.769793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da151f-773e-4e2e-ba5e-7ee69ec461d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.864633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f1490-fca5-4063-8780-40865e975579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:02.866281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f1490-fca5-4063-8780-40865e975579 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:02.891232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 816/1614] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:02.892532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff16f90-99cb-4887-9222-3a72833ce6e9 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.892819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff16f90-99cb-4887-9222-3a72833ce6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.893095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff16f90-99cb-4887-9222-3a72833ce6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.930278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4ff16f90-99cb-4887-9222-3a72833ce6e9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:02.930278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4ff16f90-99cb-4887-9222-3a72833ce6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:02.942625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 817/1615] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:02.944211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74193cc9-1bd6-4d83-8bd8-4da352547b25 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.944647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74193cc9-1bd6-4d83-8bd8-4da352547b25 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.945021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74193cc9-1bd6-4d83-8bd8-4da352547b25 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.947215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 818/1616] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 24 17:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:02.948957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880049-370f-48d7-aede-c5cffb5650ab None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.949349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880049-370f-48d7-aede-c5cffb5650ab None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.949771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880049-370f-48d7-aede-c5cffb5650ab None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.951663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 819/1617] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 24 17:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:02.957800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32bf8d-b6ac-47f7-892a-eb9d6c358fea None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:02.960536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32bf8d-b6ac-47f7-892a-eb9d6c358fea None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:02.960898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32bf8d-b6ac-47f7-892a-eb9d6c358fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:02.983242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da151f-773e-4e2e-ba5e-7ee69ec461d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:02.985742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da151f-773e-4e2e-ba5e-7ee69ec461d7 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:03.012067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 799/1618] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:03.013490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f45fee-e706-416a-a1be-3616dd16746c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.013835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f45fee-e706-416a-a1be-3616dd16746c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.014136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f45fee-e706-416a-a1be-3616dd16746c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.048406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-2a32bf8d-b6ac-47f7-892a-eb9d6c358fea None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:03.050234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2a32bf8d-b6ac-47f7-892a-eb9d6c358fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:03.051931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f0f45fee-e706-416a-a1be-3616dd16746c None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:03.053316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f0f45fee-e706-416a-a1be-3616dd16746c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:03.122892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:03.126779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:03.161164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 820/1619] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 24 17:49:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:03.162293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32bf8d-b6ac-47f7-892a-eb9d6c358fea None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:03.173905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32bf8d-b6ac-47f7-892a-eb9d6c358fea None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:03.178308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c64405-f18c-4e94-b36a-259f54a62840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:03.178929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64405-f18c-4e94-b36a-259f54a62840 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.179207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64405-f18c-4e94-b36a-259f54a62840 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.179797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64405-f18c-4e94-b36a-259f54a62840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.192738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c64405-f18c-4e94-b36a-259f54a62840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:03.194913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c64405-f18c-4e94-b36a-259f54a62840 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:03.195843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 800/1620] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 24 17:49:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:03.196755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f45fee-e706-416a-a1be-3616dd16746c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:03.209049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 821/1621] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:03.211465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c64405-f18c-4e94-b36a-259f54a62840 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:03.215066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f45fee-e706-416a-a1be-3616dd16746c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:03.221800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51670f1-e6af-468b-8802-7df68368f15f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:03.224264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51670f1-e6af-468b-8802-7df68368f15f service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.225075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51670f1-e6af-468b-8802-7df68368f15f service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.225913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51670f1-e6af-468b-8802-7df68368f15f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.240588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c64405-f18c-4e94-b36a-259f54a62840 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:03.260884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa7afe-9760-463b-bb0b-578ca24f907f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:03.261628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa7afe-9760-463b-bb0b-578ca24f907f service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.262780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa7afe-9760-463b-bb0b-578ca24f907f service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.263071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa7afe-9760-463b-bb0b-578ca24f907f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.290240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa7afe-9760-463b-bb0b-578ca24f907f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:03.291986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa7afe-9760-463b-bb0b-578ca24f907f service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:03.317410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 822/1622] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:03.319190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51670f1-e6af-468b-8802-7df68368f15f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:03.321555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51670f1-e6af-468b-8802-7df68368f15f service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:03.332681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 801/1623] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:03.593980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1d182-6adc-4b6a-9456-b64522dfa72f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.594550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1d182-6adc-4b6a-9456-b64522dfa72f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.595548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1d182-6adc-4b6a-9456-b64522dfa72f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.597100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 823/1624] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 24 17:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:03.604846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ed8df-b6a9-4106-9dd6-76fd4f10097d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.605132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ed8df-b6a9-4106-9dd6-76fd4f10097d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.605413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ed8df-b6a9-4106-9dd6-76fd4f10097d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.605805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6392-fd25-4dff-9e60-87de40608c3f None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.606151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6392-fd25-4dff-9e60-87de40608c3f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.606432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6392-fd25-4dff-9e60-87de40608c3f None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.607853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 824/1625] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 24 17:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:49:03.614653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11d8d9-16c4-4f87-b307-f2f0f3387473 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.614944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11d8d9-16c4-4f87-b307-f2f0f3387473 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.615227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11d8d9-16c4-4f87-b307-f2f0f3387473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.647319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6f8ed8df-b6a9-4106-9dd6-76fd4f10097d None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:03.650527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6f8ed8df-b6a9-4106-9dd6-76fd4f10097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:03.654816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ae11d8d9-16c4-4f87-b307-f2f0f3387473 None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:03.657149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ae11d8d9-16c4-4f87-b307-f2f0f3387473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:03.692696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 802/1626] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:49:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:03.694733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ed8df-b6a9-4106-9dd6-76fd4f10097d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:03.701919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 825/1627] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:49:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:03.717239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11d8d9-16c4-4f87-b307-f2f0f3387473 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:03.839147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ed8df-b6a9-4106-9dd6-76fd4f10097d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:03.845140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a557e-6eef-4b5d-aa3e-c86eecd55670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:03.846354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a557e-6eef-4b5d-aa3e-c86eecd55670 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.846638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a557e-6eef-4b5d-aa3e-c86eecd55670 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.847087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a557e-6eef-4b5d-aa3e-c86eecd55670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.854919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11d8d9-16c4-4f87-b307-f2f0f3387473 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:03.862352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a557e-6eef-4b5d-aa3e-c86eecd55670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:03.864948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73beaaf2-8d5c-4a51-895d-06f486073efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:03.865097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a557e-6eef-4b5d-aa3e-c86eecd55670 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:03.865638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beaaf2-8d5c-4a51-895d-06f486073efb service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.865978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beaaf2-8d5c-4a51-895d-06f486073efb service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.866292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beaaf2-8d5c-4a51-895d-06f486073efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.883853 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73beaaf2-8d5c-4a51-895d-06f486073efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:03.886427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73beaaf2-8d5c-4a51-895d-06f486073efb service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:03.892344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 803/1628] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:03.906003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 826/1629] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:06.048118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adef8c-830f-4728-b8f5-d05ec5880b9d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.048634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adef8c-830f-4728-b8f5-d05ec5880b9d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.052760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adef8c-830f-4728-b8f5-d05ec5880b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.112358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-19adef8c-830f-4728-b8f5-d05ec5880b9d None None] MFA Rules not processed for user `09774494757f4c56a077e58c1a09eacd`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:06.114361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-19adef8c-830f-4728-b8f5-d05ec5880b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:06.304248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:06.304248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 804/1630] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:06.305562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73beaaf2-8d5c-4a51-895d-06f486073efb service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.332626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73beaaf2-8d5c-4a51-895d-06f486073efb service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.360131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93555f3-8b28-4d90-8293-6b74739955dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.360980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93555f3-8b28-4d90-8293-6b74739955dc admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.361327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93555f3-8b28-4d90-8293-6b74739955dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.361836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93555f3-8b28-4d90-8293-6b74739955dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.362687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93555f3-8b28-4d90-8293-6b74739955dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.365510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93555f3-8b28-4d90-8293-6b74739955dc admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.571161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 827/1631] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/projects => generated 403 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:06.571910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adef8c-830f-4728-b8f5-d05ec5880b9d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.591880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adef8c-830f-4728-b8f5-d05ec5880b9d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.800621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffffc0c-91c3-473f-96c7-449263163138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.801052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffffc0c-91c3-473f-96c7-449263163138 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.814361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffffc0c-91c3-473f-96c7-449263163138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.814361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffffc0c-91c3-473f-96c7-449263163138 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.814361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffffc0c-91c3-473f-96c7-449263163138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.814361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffffc0c-91c3-473f-96c7-449263163138 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.933809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 805/1632] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/users => generated 308 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:06.937537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93555f3-8b28-4d90-8293-6b74739955dc admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.951939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93555f3-8b28-4d90-8293-6b74739955dc admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.964167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffdaac-277c-4444-94f9-4ccf763a18b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.965125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffdaac-277c-4444-94f9-4ccf763a18b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.965260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffdaac-277c-4444-94f9-4ccf763a18b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.965519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffdaac-277c-4444-94f9-4ccf763a18b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.966233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffdaac-277c-4444-94f9-4ccf763a18b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.967958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffdaac-277c-4444-94f9-4ccf763a18b6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.990516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 828/1633] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:06.995217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffffc0c-91c3-473f-96c7-449263163138 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.010253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffffc0c-91c3-473f-96c7-449263163138 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.023783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7faf15-99c9-4530-9ae1-f5c7df41ac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.024348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7faf15-99c9-4530-9ae1-f5c7df41ac4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.024631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7faf15-99c9-4530-9ae1-f5c7df41ac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.024906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7faf15-99c9-4530-9ae1-f5c7df41ac4c admin admin] PATH_INFO: `/v3/projects/159adfdbdf05482ca799fa76f4814ee3/users/62016dd039054c13a61cb63047970059/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.061623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7faf15-99c9-4530-9ae1-f5c7df41ac4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=159adfdbdf05482ca799fa76f4814ee3, user_id=62016dd039054c13a61cb63047970059, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.072402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7faf15-99c9-4530-9ae1-f5c7df41ac4c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.163875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 806/1634] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:06 2022] PUT /identity/v3/projects/159adfdbdf05482ca799fa76f4814ee3/users/62016dd039054c13a61cb63047970059/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:07.167145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffdaac-277c-4444-94f9-4ccf763a18b6 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.180815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffdaac-277c-4444-94f9-4ccf763a18b6 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.287466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93e276-f939-4d86-ad39-9b39f2b44420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.288302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93e276-f939-4d86-ad39-9b39f2b44420 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.288743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93e276-f939-4d86-ad39-9b39f2b44420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.289049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93e276-f939-4d86-ad39-9b39f2b44420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.290599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93e276-f939-4d86-ad39-9b39f2b44420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.294664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93e276-f939-4d86-ad39-9b39f2b44420 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.321425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 829/1635] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:07.325804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7faf15-99c9-4530-9ae1-f5c7df41ac4c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.342075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7faf15-99c9-4530-9ae1-f5c7df41ac4c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.354504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b09b05-c282-4799-8b9c-47fffaddf106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.356381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09b05-c282-4799-8b9c-47fffaddf106 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.356767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09b05-c282-4799-8b9c-47fffaddf106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.357041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09b05-c282-4799-8b9c-47fffaddf106 admin admin] PATH_INFO: `/v3/projects/159adfdbdf05482ca799fa76f4814ee3/users/62016dd039054c13a61cb63047970059/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.363434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b09b05-c282-4799-8b9c-47fffaddf106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=159adfdbdf05482ca799fa76f4814ee3, user_id=62016dd039054c13a61cb63047970059, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.373176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b09b05-c282-4799-8b9c-47fffaddf106 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.408415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 807/1636] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:07 2022] PUT /identity/v3/projects/159adfdbdf05482ca799fa76f4814ee3/users/62016dd039054c13a61cb63047970059/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:07.920876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93e276-f939-4d86-ad39-9b39f2b44420 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.933709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93e276-f939-4d86-ad39-9b39f2b44420 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.953022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:08.087854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:08.089389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:08.089837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:08.090289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] PATH_INFO: `/v3/users/e8e82fcb6d7549b1b3a62e346e661945` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:08.092202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e82fcb6d7549b1b3a62e346e661945)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:08.093427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: WARNING py.warnings [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:49:08.093427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: warnings.warn(msg) Aug 24 17:49:08.093427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: Aug 24 17:49:08.096011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:08.393053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e82fcb6d7549b1b3a62e346e661945'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.419488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e82fcb6d7549b1b3a62e346e661945'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.576385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e82fcb6d7549b1b3a62e346e661945'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.577139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e82fcb6d7549b1b3a62e346e661945'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.577801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e82fcb6d7549b1b3a62e346e661945'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.614906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 830/1637] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:07 2022] DELETE /identity/v3/users/e8e82fcb6d7549b1b3a62e346e661945 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:08.622365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd1118-8f13-4586-843f-25ff7844710b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:08.623339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd1118-8f13-4586-843f-25ff7844710b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:08.623833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd1118-8f13-4586-843f-25ff7844710b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:08.659367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-fbbd1118-8f13-4586-843f-25ff7844710b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:08.666243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-fbbd1118-8f13-4586-843f-25ff7844710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:08.750043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 808/1638] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:08.760565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:08.762046 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:08.884088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34fb65-ed47-4190-b97f-1d9f31625585 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:08.889774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b7ba-e9d1-40f0-9f26-7539e9f21ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:08.890763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b7ba-e9d1-40f0-9f26-7539e9f21ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:08.891456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b7ba-e9d1-40f0-9f26-7539e9f21ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:08.891926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b7ba-e9d1-40f0-9f26-7539e9f21ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:08.922653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0b7ba-e9d1-40f0-9f26-7539e9f21ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:08.926291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0b7ba-e9d1-40f0-9f26-7539e9f21ef2 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:08.943408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 831/1639] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:10.449838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd1118-8f13-4586-843f-25ff7844710b None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.452305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-fbbd1118-8f13-4586-843f-25ff7844710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:10.479860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd1118-8f13-4586-843f-25ff7844710b None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.486761 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.487603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.487893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.488159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] PATH_INFO: `/v3/projects/de73a596283345f2b6b87ce076767b78` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.490939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] RBAC: Authorizing `identity:delete_project(project_id=de73a596283345f2b6b87ce076767b78)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.492509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: WARNING py.warnings [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:49:10.492509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: warnings.warn(msg) Aug 24 17:49:10.492509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: Aug 24 17:49:10.494177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.718579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:10.731471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de73a596283345f2b6b87ce076767b78', 'request_id': 'req-76617956-468e-4bfb-b29a-9aa8393bbfae'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:10.836920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de73a596283345f2b6b87ce076767b78', 'request_id': 'req-76617956-468e-4bfb-b29a-9aa8393bbfae'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:10.850716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de73a596283345f2b6b87ce076767b78', 'request_id': 'req-76617956-468e-4bfb-b29a-9aa8393bbfae'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:10.912989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] The token cache is being invalidate because project de73a596283345f2b6b87ce076767b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:10.913673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:10.914627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76617956-468e-4bfb-b29a-9aa8393bbfae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:10.916635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 809/1640] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:10 2022] DELETE /identity/v3/projects/de73a596283345f2b6b87ce076767b78 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:10.922149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4740d0-f5db-41b3-ac2e-1c8e8589c180 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.922628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4740d0-f5db-41b3-ac2e-1c8e8589c180 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.922698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4740d0-f5db-41b3-ac2e-1c8e8589c180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.978438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-bb4740d0-f5db-41b3-ac2e-1c8e8589c180 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:10.979962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bb4740d0-f5db-41b3-ac2e-1c8e8589c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:11.108138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:11.247321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 832/1641] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:11.252432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40dda7-0b77-4ed4-9430-6a51d1e96008 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:11.252723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40dda7-0b77-4ed4-9430-6a51d1e96008 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:11.253232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40dda7-0b77-4ed4-9430-6a51d1e96008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:11.314593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6a40dda7-0b77-4ed4-9430-6a51d1e96008 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:11.317059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6a40dda7-0b77-4ed4-9430-6a51d1e96008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:11.343444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 810/1642] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:11.344637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4740d0-f5db-41b3-ac2e-1c8e8589c180 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:11.367567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4740d0-f5db-41b3-ac2e-1c8e8589c180 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:11.380798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:11.381577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:11.382124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:11.382665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:11.385094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:11.390733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:11.392461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:11.426190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 833/1643] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:11.430526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40dda7-0b77-4ed4-9430-6a51d1e96008 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:11.448605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40dda7-0b77-4ed4-9430-6a51d1e96008 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:11.461985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db7ab9-10b4-4229-b47d-bd535a529f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:11.462611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db7ab9-10b4-4229-b47d-bd535a529f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:11.463046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db7ab9-10b4-4229-b47d-bd535a529f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:11.463498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db7ab9-10b4-4229-b47d-bd535a529f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:11.464331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db7ab9-10b4-4229-b47d-bd535a529f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:11.466553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db7ab9-10b4-4229-b47d-bd535a529f85 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:11.652458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 811/1644] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:11 2022] POST /identity/v3/projects => generated 403 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:11.657135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:11.677528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc0ed6-c207-428b-b1ef-ab0c7977be91 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:11.836497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdbe4c-dbc7-4e5b-938d-fa37276f57d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:11.837203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdbe4c-dbc7-4e5b-938d-fa37276f57d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:11.837486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdbe4c-dbc7-4e5b-938d-fa37276f57d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:11.838165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdbe4c-dbc7-4e5b-938d-fa37276f57d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:11.839015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdbe4c-dbc7-4e5b-938d-fa37276f57d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:11.841142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdbe4c-dbc7-4e5b-938d-fa37276f57d7 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:12.002594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 834/1645] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:11 2022] POST /identity/v3/users => generated 302 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:12.006388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db7ab9-10b4-4229-b47d-bd535a529f85 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:12.024274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db7ab9-10b4-4229-b47d-bd535a529f85 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:12.047716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e37143-c05b-486c-8976-5cde3c8d96d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:12.048448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e37143-c05b-486c-8976-5cde3c8d96d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:12.048805 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e37143-c05b-486c-8976-5cde3c8d96d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:12.049271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e37143-c05b-486c-8976-5cde3c8d96d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:12.053556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e37143-c05b-486c-8976-5cde3c8d96d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:12.059359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e37143-c05b-486c-8976-5cde3c8d96d7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:12.127461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 812/1646] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:12.132121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdbe4c-dbc7-4e5b-938d-fa37276f57d7 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:12.154519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdbe4c-dbc7-4e5b-938d-fa37276f57d7 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:12.172949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cf836-875a-4e47-a064-832d2eda2f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:12.173646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cf836-875a-4e47-a064-832d2eda2f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:12.173960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cf836-875a-4e47-a064-832d2eda2f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:12.178279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cf836-875a-4e47-a064-832d2eda2f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:12.179155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cf836-875a-4e47-a064-832d2eda2f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:12.190329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cf836-875a-4e47-a064-832d2eda2f6c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:12.242799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 835/1647] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:12.246648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e37143-c05b-486c-8976-5cde3c8d96d7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:12.260936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e37143-c05b-486c-8976-5cde3c8d96d7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:12.272925 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12f8d8-64b2-4c2c-84ef-470c507a4439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:12.278097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12f8d8-64b2-4c2c-84ef-470c507a4439 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:12.278621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12f8d8-64b2-4c2c-84ef-470c507a4439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:12.279046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12f8d8-64b2-4c2c-84ef-470c507a4439 admin admin] PATH_INFO: `/v3/projects/eba07b03fee64c09a26479efad760fdd/users/ac010ed10e314aab8a6e5a26717db4c8/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:12.312985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12f8d8-64b2-4c2c-84ef-470c507a4439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba07b03fee64c09a26479efad760fdd, user_id=ac010ed10e314aab8a6e5a26717db4c8, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:12.318599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12f8d8-64b2-4c2c-84ef-470c507a4439 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:12.520416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 813/1648] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:12 2022] PUT /identity/v3/projects/eba07b03fee64c09a26479efad760fdd/users/ac010ed10e314aab8a6e5a26717db4c8/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:12.521107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761b4b1-ff2c-48eb-b2bb-16162ad66546 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:12.521107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761b4b1-ff2c-48eb-b2bb-16162ad66546 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:12.521107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761b4b1-ff2c-48eb-b2bb-16162ad66546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:12.623655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f761b4b1-ff2c-48eb-b2bb-16162ad66546 None None] MFA Rules not processed for user `ac010ed10e314aab8a6e5a26717db4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:12.624942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f761b4b1-ff2c-48eb-b2bb-16162ad66546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:12.788378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12f8d8-64b2-4c2c-84ef-470c507a4439 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:12.790863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 836/1649] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:12 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:12.918517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89ff48-ba1a-4715-b558-9d9af1c42041 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:12.919800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89ff48-ba1a-4715-b558-9d9af1c42041 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:12.920289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89ff48-ba1a-4715-b558-9d9af1c42041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:12.942669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12f8d8-64b2-4c2c-84ef-470c507a4439 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:12.952121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e65a96-9ff8-4c39-b5f1-9d6963fae2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:12.953150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e65a96-9ff8-4c39-b5f1-9d6963fae2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:12.953762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e65a96-9ff8-4c39-b5f1-9d6963fae2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:12.954261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e65a96-9ff8-4c39-b5f1-9d6963fae2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:12.955193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e65a96-9ff8-4c39-b5f1-9d6963fae2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:12.960339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e65a96-9ff8-4c39-b5f1-9d6963fae2d9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:12.986135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7a89ff48-ba1a-4715-b558-9d9af1c42041 None None] MFA Rules not processed for user `62016dd039054c13a61cb63047970059`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:12.988008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7a89ff48-ba1a-4715-b558-9d9af1c42041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:13.020694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:13.094631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 814/1650] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:12 2022] POST /identity/v3/projects => generated 403 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:13.096181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e65a96-9ff8-4c39-b5f1-9d6963fae2d9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:13.107263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 837/1651] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:12 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:13.254101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e65a96-9ff8-4c39-b5f1-9d6963fae2d9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:13.255877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:13.268832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bbea4-8aaf-45eb-ab3a-e69c50118939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:13.269643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bbea4-8aaf-45eb-ab3a-e69c50118939 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:13.269963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bbea4-8aaf-45eb-ab3a-e69c50118939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:13.270243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bbea4-8aaf-45eb-ab3a-e69c50118939 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:13.271151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bbea4-8aaf-45eb-ab3a-e69c50118939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:13.273217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bbea4-8aaf-45eb-ab3a-e69c50118939 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:13.453556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 815/1652] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:13 2022] POST /identity/v3/users => generated 308 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:13.458524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89ff48-ba1a-4715-b558-9d9af1c42041 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:13.474841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89ff48-ba1a-4715-b558-9d9af1c42041 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:13.493819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad008292-e0e1-45c2-bf55-c27f003a5d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:13.493819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad008292-e0e1-45c2-bf55-c27f003a5d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:13.493819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad008292-e0e1-45c2-bf55-c27f003a5d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:13.493819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad008292-e0e1-45c2-bf55-c27f003a5d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:13.494547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad008292-e0e1-45c2-bf55-c27f003a5d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:13.496272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad008292-e0e1-45c2-bf55-c27f003a5d8e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:13.524474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 838/1653] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:13.528794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bbea4-8aaf-45eb-ab3a-e69c50118939 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:13.542931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bbea4-8aaf-45eb-ab3a-e69c50118939 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:13.566688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95042e-8ffb-4884-8ee8-3dad2f9febdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:13.567385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95042e-8ffb-4884-8ee8-3dad2f9febdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:13.567673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95042e-8ffb-4884-8ee8-3dad2f9febdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:13.567955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95042e-8ffb-4884-8ee8-3dad2f9febdc admin admin] PATH_INFO: `/v3/projects/76ceba619360484a883c2f5eb49bf6f6/users/12611970228f4aebb623d9e1f04d07f2/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:13.611495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95042e-8ffb-4884-8ee8-3dad2f9febdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ceba619360484a883c2f5eb49bf6f6, user_id=12611970228f4aebb623d9e1f04d07f2, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:13.622859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95042e-8ffb-4884-8ee8-3dad2f9febdc admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:13.734364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 816/1654] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:13 2022] PUT /identity/v3/projects/76ceba619360484a883c2f5eb49bf6f6/users/12611970228f4aebb623d9e1f04d07f2/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:13.739021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad008292-e0e1-45c2-bf55-c27f003a5d8e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:13.759781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad008292-e0e1-45c2-bf55-c27f003a5d8e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:13.852785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe83cb-df69-4672-9c51-eefcd594b9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:13.853312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe83cb-df69-4672-9c51-eefcd594b9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:13.853598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe83cb-df69-4672-9c51-eefcd594b9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:13.853895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe83cb-df69-4672-9c51-eefcd594b9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:13.854559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe83cb-df69-4672-9c51-eefcd594b9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:13.856224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe83cb-df69-4672-9c51-eefcd594b9c3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:13.897994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 839/1655] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:13.899366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95042e-8ffb-4884-8ee8-3dad2f9febdc admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:13.921059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95042e-8ffb-4884-8ee8-3dad2f9febdc admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:13.931038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:13.931795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:13.932176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:13.932579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] PATH_INFO: `/v3/projects/76ceba619360484a883c2f5eb49bf6f6/users/12611970228f4aebb623d9e1f04d07f2/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:13.940028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ceba619360484a883c2f5eb49bf6f6, user_id=12611970228f4aebb623d9e1f04d07f2, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:13.943846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:13.987046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 817/1656] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:13 2022] PUT /identity/v3/projects/76ceba619360484a883c2f5eb49bf6f6/users/12611970228f4aebb623d9e1f04d07f2/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:13.990714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983ca88-6502-41a2-bf61-382bd95e700f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:13.990849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983ca88-6502-41a2-bf61-382bd95e700f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:13.991477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983ca88-6502-41a2-bf61-382bd95e700f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:14.118491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3983ca88-6502-41a2-bf61-382bd95e700f None None] MFA Rules not processed for user `12611970228f4aebb623d9e1f04d07f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:14.120902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3983ca88-6502-41a2-bf61-382bd95e700f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:14.260105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 840/1657] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:13 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:14.282296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:14.287578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:14.390778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05868ab1-64f6-4e18-a15f-5cc8f222c740 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:14.396179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d0683-b8c4-4c1e-9e27-61f486fcde74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:14.397136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0683-b8c4-4c1e-9e27-61f486fcde74 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:14.398017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0683-b8c4-4c1e-9e27-61f486fcde74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:14.398297 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0683-b8c4-4c1e-9e27-61f486fcde74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:14.408887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d0683-b8c4-4c1e-9e27-61f486fcde74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:14.411205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d0683-b8c4-4c1e-9e27-61f486fcde74 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:14.436429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 818/1658] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:14 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:14.642938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983ca88-6502-41a2-bf61-382bd95e700f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:14.707608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:14.807287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983ca88-6502-41a2-bf61-382bd95e700f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:14.821876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:14.822546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:14.822825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:14.823101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] PATH_INFO: `/v3/users/ac010ed10e314aab8a6e5a26717db4c8` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:14.824302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac010ed10e314aab8a6e5a26717db4c8)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:14.826336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:15.094893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc22d9-c2ba-4bca-8327-25a78c40cab9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.095830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc22d9-c2ba-4bca-8327-25a78c40cab9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.096249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc22d9-c2ba-4bca-8327-25a78c40cab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.126413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac010ed10e314aab8a6e5a26717db4c8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.131694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-45bc22d9-c2ba-4bca-8327-25a78c40cab9 None None] MFA Rules not processed for user `f5fcd46b400942a99dafdba6327ba49e`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:15.134564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-45bc22d9-c2ba-4bca-8327-25a78c40cab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.149723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac010ed10e314aab8a6e5a26717db4c8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.250925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac010ed10e314aab8a6e5a26717db4c8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.251944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac010ed10e314aab8a6e5a26717db4c8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.253415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5efe26e1-4c07-429a-92ef-cd602ae73582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac010ed10e314aab8a6e5a26717db4c8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.301880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 841/1659] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:14 2022] DELETE /identity/v3/users/ac010ed10e314aab8a6e5a26717db4c8 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:15.303729 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9340-d9c5-4272-95dc-7bf32755a576 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.304033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9340-d9c5-4272-95dc-7bf32755a576 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.304309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9340-d9c5-4272-95dc-7bf32755a576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.336455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3ddc9340-d9c5-4272-95dc-7bf32755a576 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:15.338062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3ddc9340-d9c5-4272-95dc-7bf32755a576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.376003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 819/1660] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:15.400189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6712b1-6bf5-490d-96df-e00a840fbd5e None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.401053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6712b1-6bf5-490d-96df-e00a840fbd5e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.403009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6712b1-6bf5-490d-96df-e00a840fbd5e None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.426472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 820/1661] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:49:15 2022] GET /identity => generated 270 bytes in 50 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:15.443331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031853c1-e972-4aa1-b78e-8726d477ab5a None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.443629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031853c1-e972-4aa1-b78e-8726d477ab5a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.446169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031853c1-e972-4aa1-b78e-8726d477ab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.486086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-031853c1-e972-4aa1-b78e-8726d477ab5a None None] MFA Rules not processed for user `f5fcd46b400942a99dafdba6327ba49e`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:15.486086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-031853c1-e972-4aa1-b78e-8726d477ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.538292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 842/1662] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:15.539792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f5c1d-e877-4b8b-a26a-22e24e42656f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.540150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f5c1d-e877-4b8b-a26a-22e24e42656f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.540595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f5c1d-e877-4b8b-a26a-22e24e42656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.577172 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-537f5c1d-e877-4b8b-a26a-22e24e42656f None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:15.580646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-537f5c1d-e877-4b8b-a26a-22e24e42656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.586429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:15.634988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 821/1663] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:15.647050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45d390-e77f-4aca-a1bd-1ff3d07b93cb None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.647334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45d390-e77f-4aca-a1bd-1ff3d07b93cb None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.647608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45d390-e77f-4aca-a1bd-1ff3d07b93cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.684311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:15.704756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-4c45d390-e77f-4aca-a1bd-1ff3d07b93cb None None] MFA Rules not processed for user `251bc85f89774142a97acd77895b14e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:15.708965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4c45d390-e77f-4aca-a1bd-1ff3d07b93cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.744847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 843/1664] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:15.752293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f5c1d-e877-4b8b-a26a-22e24e42656f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:15.758071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-537f5c1d-e877-4b8b-a26a-22e24e42656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.806394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f5c1d-e877-4b8b-a26a-22e24e42656f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:15.820025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:15.820742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.821047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.821683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] PATH_INFO: `/v3/users/181725023fc14d8e9d077d6012eb2e22` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.825098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=181725023fc14d8e9d077d6012eb2e22)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:15.829610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:15.893360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 822/1665] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:15.898114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45d390-e77f-4aca-a1bd-1ff3d07b93cb None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:15.903775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4c45d390-e77f-4aca-a1bd-1ff3d07b93cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.995962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45d390-e77f-4aca-a1bd-1ff3d07b93cb None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:16.002301 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:16.003543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.004736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.005190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.023883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:16.029538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:16.061920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 823/1666] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:16.063033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:16.064737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.150719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181725023fc14d8e9d077d6012eb2e22'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.173517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181725023fc14d8e9d077d6012eb2e22'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.235476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c58e4-d23a-4af5-8b8e-05f2337c0f8a service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:16.245391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da45fa-0edc-4dce-b89d-540cd2219876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:16.246992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da45fa-0edc-4dce-b89d-540cd2219876 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.246992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da45fa-0edc-4dce-b89d-540cd2219876 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.247552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da45fa-0edc-4dce-b89d-540cd2219876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.264232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da45fa-0edc-4dce-b89d-540cd2219876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:16.267564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da45fa-0edc-4dce-b89d-540cd2219876 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:16.284822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 824/1667] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:16.286687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9dda4-fc28-42f1-b5ba-ed40f2b99665 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.287574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9dda4-fc28-42f1-b5ba-ed40f2b99665 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.288384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9dda4-fc28-42f1-b5ba-ed40f2b99665 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.292898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181725023fc14d8e9d077d6012eb2e22'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.292898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181725023fc14d8e9d077d6012eb2e22'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.293600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-805e8f30-8f00-4be9-bcca-a2eef155c5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181725023fc14d8e9d077d6012eb2e22'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.308726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 825/1668] 10.222.0.76 () {54 vars in 961 bytes} [Wed Aug 24 17:49:16 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:16.310375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9dda4-fc28-42f1-b5ba-ed40f2b99665 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:16.314504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-95f9dda4-fc28-42f1-b5ba-ed40f2b99665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.326420 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 844/1669] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:15 2022] DELETE /identity/v3/users/181725023fc14d8e9d077d6012eb2e22 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:16.328522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489990-5ca3-42d1-a2c3-e4a50d4ca405 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.328829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489990-5ca3-42d1-a2c3-e4a50d4ca405 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.330114 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489990-5ca3-42d1-a2c3-e4a50d4ca405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.387307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1c489990-5ca3-42d1-a2c3-e4a50d4ca405 None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:16.388834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1c489990-5ca3-42d1-a2c3-e4a50d4ca405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.464146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:16.554407 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9dda4-fc28-42f1-b5ba-ed40f2b99665 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:16.594858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:16.598575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.599530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.599882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.606769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.625073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 845/1670] 10.222.0.76 () {58 vars in 1058 bytes} [Wed Aug 24 17:49:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:16.630952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ebf62-e47c-486f-a0b4-44768b09ef55 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.631484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ebf62-e47c-486f-a0b4-44768b09ef55 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.631828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ebf62-e47c-486f-a0b4-44768b09ef55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.666078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-cb5ebf62-e47c-486f-a0b4-44768b09ef55 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:16.668816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cb5ebf62-e47c-486f-a0b4-44768b09ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.802602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:16.808813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:16.867833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 826/1671] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:16.868558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:16.906437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c36b4-882f-4313-b089-a52a73ddc0a9 service nova] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:16.923886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:16.924496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.925339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.925638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.937099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.938864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 846/1672] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:16.975084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ebf62-e47c-486f-a0b4-44768b09ef55 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:16.975833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:16.977308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cb5ebf62-e47c-486f-a0b4-44768b09ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.984644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.009252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 827/1673] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:17.209612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ebf62-e47c-486f-a0b4-44768b09ef55 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.219982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758689-b00c-4fd7-8b53-aac6bfc43fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.233789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758689-b00c-4fd7-8b53-aac6bfc43fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.234633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758689-b00c-4fd7-8b53-aac6bfc43fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.235732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758689-b00c-4fd7-8b53-aac6bfc43fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.271140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a758689-b00c-4fd7-8b53-aac6bfc43fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.274865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a758689-b00c-4fd7-8b53-aac6bfc43fa2 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.290116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 847/1674] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:17.441831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:17.461372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.464995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:17.496630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289145-e369-4d6b-9e5e-9a3d45831adf service placement] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.515067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.517974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.522490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.522893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] PATH_INFO: `/v3/projects/eba07b03fee64c09a26479efad760fdd` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.525536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eba07b03fee64c09a26479efad760fdd)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.533376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.732678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eba07b03fee64c09a26479efad760fdd', 'request_id': 'req-1d5407e0-6228-43d5-a9db-62484df88bc8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.890601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eba07b03fee64c09a26479efad760fdd', 'request_id': 'req-1d5407e0-6228-43d5-a9db-62484df88bc8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.900282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eba07b03fee64c09a26479efad760fdd', 'request_id': 'req-1d5407e0-6228-43d5-a9db-62484df88bc8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.928494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] The token cache is being invalidate because project eba07b03fee64c09a26479efad760fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:17.928883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.934390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1d5407e0-6228-43d5-a9db-62484df88bc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.936191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 828/1675] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:17 2022] DELETE /identity/v3/projects/eba07b03fee64c09a26479efad760fdd => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:17.940365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758689-b00c-4fd7-8b53-aac6bfc43fa2 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.954034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758689-b00c-4fd7-8b53-aac6bfc43fa2 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.008159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce38f5-94bf-4f7d-b0db-46896f63377d None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.008366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce38f5-94bf-4f7d-b0db-46896f63377d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.008692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce38f5-94bf-4f7d-b0db-46896f63377d None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.010739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 829/1676] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 24 17:49:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:18.035206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce38f5-94bf-4f7d-b0db-46896f63377d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.043313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-04ce38f5-94bf-4f7d-b0db-46896f63377d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:18.076205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:18.272560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce38f5-94bf-4f7d-b0db-46896f63377d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.280444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.281475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.281948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.282195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] PATH_INFO: `/v3/users/12611970228f4aebb623d9e1f04d07f2` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.284385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12611970228f4aebb623d9e1f04d07f2)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.285386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.289249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.289516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.289831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.290150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] PATH_INFO: `/v3/users/d9aa128932f94a57a7b0bdf3f18ed1bf` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.293649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9aa128932f94a57a7b0bdf3f18ed1bf)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.302696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.593836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12611970228f4aebb623d9e1f04d07f2'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.603823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9aa128932f94a57a7b0bdf3f18ed1bf'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.626062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12611970228f4aebb623d9e1f04d07f2'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.714660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9aa128932f94a57a7b0bdf3f18ed1bf'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.717668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9aa128932f94a57a7b0bdf3f18ed1bf'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.750212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9aa128932f94a57a7b0bdf3f18ed1bf'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.751873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-99a7a66f-84a0-4f60-873d-898693f1935b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9aa128932f94a57a7b0bdf3f18ed1bf'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.787732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12611970228f4aebb623d9e1f04d07f2'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.788427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12611970228f4aebb623d9e1f04d07f2'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.789066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f2665482-5242-4871-9f65-49dd02ef3588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12611970228f4aebb623d9e1f04d07f2'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:18.822438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 848/1677] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:17 2022] DELETE /identity/v3/users/12611970228f4aebb623d9e1f04d07f2 => generated 0 bytes in 882 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:18.823873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a365879-2a9f-4fca-a467-f245e792b3fe None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.824176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a365879-2a9f-4fca-a467-f245e792b3fe None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.824468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a365879-2a9f-4fca-a467-f245e792b3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.830733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 830/1678] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:18 2022] DELETE /identity/v3/users/d9aa128932f94a57a7b0bdf3f18ed1bf => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:18.836024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2c0b-f788-45bb-9edd-91a8adb6af1d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.836282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2c0b-f788-45bb-9edd-91a8adb6af1d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.836556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2c0b-f788-45bb-9edd-91a8adb6af1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.888817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-83ae2c0b-f788-45bb-9edd-91a8adb6af1d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:18.891330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-83ae2c0b-f788-45bb-9edd-91a8adb6af1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:18.968887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-0a365879-2a9f-4fca-a467-f245e792b3fe None None] MFA Rules not processed for user `efc068011c14492d9ba483650cf632e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:18.976369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0a365879-2a9f-4fca-a467-f245e792b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:18.988692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:19.085311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 831/1679] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:19.122972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae2c0b-f788-45bb-9edd-91a8adb6af1d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:19.125373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-83ae2c0b-f788-45bb-9edd-91a8adb6af1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:19.140841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 849/1680] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:19.160020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a365879-2a9f-4fca-a467-f245e792b3fe None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:19.168863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae2c0b-f788-45bb-9edd-91a8adb6af1d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:19.178004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0a365879-2a9f-4fca-a467-f245e792b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:19.184431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:19.188647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:19.188647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:19.188647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] PATH_INFO: `/v3/users/09774494757f4c56a077e58c1a09eacd` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:19.189315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09774494757f4c56a077e58c1a09eacd)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:19.192520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:19.216309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a365879-2a9f-4fca-a467-f245e792b3fe None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:19.228339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:19.231624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:19.233064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:19.233361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:19.262965 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:19.264946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:19.301699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 850/1681] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:19.305887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:19.305887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:19.328470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2429b10-dc9a-4441-b83c-bd4200e2b725 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:19.350227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:19.350949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:19.351459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:19.351818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:19.368292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:19.370093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:19.381901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 851/1682] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:19.462703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09774494757f4c56a077e58c1a09eacd'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.491772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:19.493976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:19.515970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ffe9f-8de1-48da-87c0-fff4a8548c79 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:19.525879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:19.526879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:19.527545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:19.528055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] PATH_INFO: `/v3/projects/49fb7ae63dac46a3883ef566a3763870` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:19.529554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49fb7ae63dac46a3883ef566a3763870)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:19.532023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:19.542262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09774494757f4c56a077e58c1a09eacd'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.543059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09774494757f4c56a077e58c1a09eacd'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.554801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09774494757f4c56a077e58c1a09eacd'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.556341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ba217c3d-5c95-4363-9e74-1886a100e03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09774494757f4c56a077e58c1a09eacd'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.614572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 832/1683] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:19 2022] DELETE /identity/v3/users/09774494757f4c56a077e58c1a09eacd => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:19.693109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:19.733766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49fb7ae63dac46a3883ef566a3763870', 'request_id': 'req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.861870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49fb7ae63dac46a3883ef566a3763870', 'request_id': 'req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.871220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49fb7ae63dac46a3883ef566a3763870', 'request_id': 'req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.908665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] The token cache is being invalidate because project 49fb7ae63dac46a3883ef566a3763870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:19.909008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.916953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc7eb038-b80f-4990-aaa7-0a0f6e2fd1e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.921893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 852/1684] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:19 2022] DELETE /identity/v3/projects/49fb7ae63dac46a3883ef566a3763870 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:19.927435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b270-6da1-4f12-911f-9e77cb17a58e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:19.927783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b270-6da1-4f12-911f-9e77cb17a58e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:19.928112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b270-6da1-4f12-911f-9e77cb17a58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:19.976625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b7c4b270-6da1-4f12-911f-9e77cb17a58e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:19.979002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b7c4b270-6da1-4f12-911f-9e77cb17a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:20.200723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 833/1685] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:20.208676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49cb3-c4b9-4c6a-ad2e-8a668727ffae None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.209014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49cb3-c4b9-4c6a-ad2e-8a668727ffae None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.209255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49cb3-c4b9-4c6a-ad2e-8a668727ffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.267152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-43b49cb3-c4b9-4c6a-ad2e-8a668727ffae None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:20.269363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-43b49cb3-c4b9-4c6a-ad2e-8a668727ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:20.300562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4b270-6da1-4f12-911f-9e77cb17a58e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.303522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 853/1686] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:20.360342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4b270-6da1-4f12-911f-9e77cb17a58e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:20.380765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:20.381503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.390325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.390676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.391560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:20.391854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:20.393690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:20.421714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 834/1687] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:20.423271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b49cb3-c4b9-4c6a-ad2e-8a668727ffae None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.435567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b49cb3-c4b9-4c6a-ad2e-8a668727ffae None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:20.441178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12f902-6736-43e9-9a75-dacffec90ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:20.441814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12f902-6736-43e9-9a75-dacffec90ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.442169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12f902-6736-43e9-9a75-dacffec90ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.442457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12f902-6736-43e9-9a75-dacffec90ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.457666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12f902-6736-43e9-9a75-dacffec90ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:20.460490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12f902-6736-43e9-9a75-dacffec90ea9 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:20.602252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 854/1688] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:20 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:20.603576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.619193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e84fa3-e270-496e-9a75-a774e2ad5705 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:20.759330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0378065-a9b0-4524-b33f-cdad9f11e42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:20.759966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0378065-a9b0-4524-b33f-cdad9f11e42c admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.760382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0378065-a9b0-4524-b33f-cdad9f11e42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.760677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0378065-a9b0-4524-b33f-cdad9f11e42c admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.761324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0378065-a9b0-4524-b33f-cdad9f11e42c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:20.763689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0378065-a9b0-4524-b33f-cdad9f11e42c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:20.860414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 835/1689] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:20 2022] POST /identity/v3/users => generated 290 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:20.861580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12f902-6736-43e9-9a75-dacffec90ea9 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.873252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12f902-6736-43e9-9a75-dacffec90ea9 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:20.878303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dd3b0-6ee0-42c1-8cdf-59e353dc4610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:20.878977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dd3b0-6ee0-42c1-8cdf-59e353dc4610 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.879264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dd3b0-6ee0-42c1-8cdf-59e353dc4610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.879736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dd3b0-6ee0-42c1-8cdf-59e353dc4610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.880544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dd3b0-6ee0-42c1-8cdf-59e353dc4610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:20.883308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dd3b0-6ee0-42c1-8cdf-59e353dc4610 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:20.901203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 855/1690] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:20.906008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0378065-a9b0-4524-b33f-cdad9f11e42c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.917118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0378065-a9b0-4524-b33f-cdad9f11e42c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:20.925633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdc381-a319-4b8d-aa7b-348d6118cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:20.926737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc381-a319-4b8d-aa7b-348d6118cd2f admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.927192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc381-a319-4b8d-aa7b-348d6118cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.927650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc381-a319-4b8d-aa7b-348d6118cd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.928557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc381-a319-4b8d-aa7b-348d6118cd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:20.934710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc381-a319-4b8d-aa7b-348d6118cd2f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:20.966571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dd3b0-6ee0-42c1-8cdf-59e353dc4610 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.971293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 836/1691] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:20.979414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dd3b0-6ee0-42c1-8cdf-59e353dc4610 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.006350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.007055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.007316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.007779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] PATH_INFO: `/v3/projects/eae14007e8564c9686c0d28f8698f342/users/ad2f353369fa4c50a108072c6bb986d7/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.053255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae14007e8564c9686c0d28f8698f342, user_id=ad2f353369fa4c50a108072c6bb986d7, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.053255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.166425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:21.184039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 856/1692] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:20 2022] PUT /identity/v3/projects/eae14007e8564c9686c0d28f8698f342/users/ad2f353369fa4c50a108072c6bb986d7/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:21.187433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea5d4a-9bf6-4734-8369-0682530d7e05 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.187813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea5d4a-9bf6-4734-8369-0682530d7e05 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.188087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea5d4a-9bf6-4734-8369-0682530d7e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.208633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.231161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.240825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:21.259450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d7ea5d4a-9bf6-4734-8369-0682530d7e05 None None] MFA Rules not processed for user `ad2f353369fa4c50a108072c6bb986d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:21.261121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d7ea5d4a-9bf6-4734-8369-0682530d7e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.344721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 837/1693] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:21.346667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35734a-70f4-41aa-87bd-edd06c0f929a None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.347221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35734a-70f4-41aa-87bd-edd06c0f929a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.347568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35734a-70f4-41aa-87bd-edd06c0f929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.382287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-5e35734a-70f4-41aa-87bd-edd06c0f929a None None] MFA Rules not processed for user `2b6b0d0cbdb54a46a7e9d6936c47cc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:21.384138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5e35734a-70f4-41aa-87bd-edd06c0f929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.524141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f517c-97cd-48fc-99f8-bcd5f9aa2e73 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.544074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.547249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.550217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.550812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] PATH_INFO: `/v3/projects/76ceba619360484a883c2f5eb49bf6f6` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.554327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ceba619360484a883c2f5eb49bf6f6)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.573581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.591699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 838/1694] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:21.592957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35734a-70f4-41aa-87bd-edd06c0f929a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.595722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5e35734a-70f4-41aa-87bd-edd06c0f929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.774210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ceba619360484a883c2f5eb49bf6f6', 'request_id': 'req-1397896a-307f-414a-b9b6-602806aaa799'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.780385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35734a-70f4-41aa-87bd-edd06c0f929a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.833192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ceba619360484a883c2f5eb49bf6f6', 'request_id': 'req-1397896a-307f-414a-b9b6-602806aaa799'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.842610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ceba619360484a883c2f5eb49bf6f6', 'request_id': 'req-1397896a-307f-414a-b9b6-602806aaa799'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.871525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] The token cache is being invalidate because project 76ceba619360484a883c2f5eb49bf6f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:21.872371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.876897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.881327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 857/1695] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:21 2022] DELETE /identity/v3/projects/76ceba619360484a883c2f5eb49bf6f6 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:21.882964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.885438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.886803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.887512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.887870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.888298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.892621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.923597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.925825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.956917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 839/1696] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:21.960961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.966188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.042785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397896a-307f-414a-b9b6-602806aaa799 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:22.058228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:22.059011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:22.059405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:22.059878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:22.063092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:22.068021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:22.124469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158f077-c0c3-45bb-810e-fe45f91aa8a8 service placement] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:22.133623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:22.136460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:22.136753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:22.137036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] PATH_INFO: `/v3/projects/4a6a1625930e40f4834c0fbf69b04cce` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:22.142277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6a1625930e40f4834c0fbf69b04cce)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:22.145129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:22.201295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 858/1697] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:21 2022] POST /identity/v3/projects => generated 381 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:22.202519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:22.206746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.359776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6a1625930e40f4834c0fbf69b04cce', 'request_id': 'req-06642eb5-fb6d-45e1-b59f-f17c281df5d7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:22.445365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1472-4c2a-424b-9ce5-b7bb6d12eb92 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:22.459340 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6a1625930e40f4834c0fbf69b04cce', 'request_id': 'req-06642eb5-fb6d-45e1-b59f-f17c281df5d7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:22.467499 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6a1625930e40f4834c0fbf69b04cce', 'request_id': 'req-06642eb5-fb6d-45e1-b59f-f17c281df5d7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:22.491314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] The token cache is being invalidate because project 4a6a1625930e40f4834c0fbf69b04cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:22.491640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:22.492408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:22.498167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 840/1698] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:21 2022] DELETE /identity/v3/projects/4a6a1625930e40f4834c0fbf69b04cce => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:22.499258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:22.502628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:22.506036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.598226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:22.618835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3b233-0f91-45ed-8109-b13aeaaa7b91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:22.620601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3b233-0f91-45ed-8109-b13aeaaa7b91 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:22.621058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3b233-0f91-45ed-8109-b13aeaaa7b91 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:22.621481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3b233-0f91-45ed-8109-b13aeaaa7b91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:22.624444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-82a3b233-0f91-45ed-8109-b13aeaaa7b91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.686117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642eb5-fb6d-45e1-b59f-f17c281df5d7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:22.702440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:22.703237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:22.703607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:22.704172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] PATH_INFO: `/v3/projects/ca875d9739724041937d23d7ca212d91` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:22.705564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca875d9739724041937d23d7ca212d91)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:22.708388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:22.774139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3b233-0f91-45ed-8109-b13aeaaa7b91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:22.777576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3b233-0f91-45ed-8109-b13aeaaa7b91 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:22.801310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 859/1699] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:22.803943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b1e99-a74c-4a9c-bb77-7b3a596e5714 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:22.804238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b1e99-a74c-4a9c-bb77-7b3a596e5714 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:22.804550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b1e99-a74c-4a9c-bb77-7b3a596e5714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:22.828267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c90b1e99-a74c-4a9c-bb77-7b3a596e5714 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:22.831756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c90b1e99-a74c-4a9c-bb77-7b3a596e5714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.840671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 860/1700] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:22.842039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b1e99-a74c-4a9c-bb77-7b3a596e5714 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:22.843600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c90b1e99-a74c-4a9c-bb77-7b3a596e5714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.858532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b1e99-a74c-4a9c-bb77-7b3a596e5714 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:22.866456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:22.866456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:22.866456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:22.866456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:22.867002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:22.868052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:22.946129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca875d9739724041937d23d7ca212d91', 'request_id': 'req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.104303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 861/1701] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:22 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:23.105273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.110295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:23.147816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca875d9739724041937d23d7ca212d91', 'request_id': 'req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.157890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3229570-8f8e-4eec-958a-fd7b91e582d0 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.165361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca875d9739724041937d23d7ca212d91', 'request_id': 'req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.198962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] The token cache is being invalidate because project ca875d9739724041937d23d7ca212d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:23.199265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.201610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-70b1eb1e-87fe-470f-8eab-081cf5b0ee6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.217772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 841/1702] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:22 2022] DELETE /identity/v3/projects/ca875d9739724041937d23d7ca212d91 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:23.221320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce409-d525-4ecc-b3b5-04d3a9b55ee0 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.222887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce409-d525-4ecc-b3b5-04d3a9b55ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.224634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce409-d525-4ecc-b3b5-04d3a9b55ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.247017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:23.282377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.284249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.284760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.285786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] PATH_INFO: `/v3/users/62016dd039054c13a61cb63047970059` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.289828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62016dd039054c13a61cb63047970059)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.298602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-cd1ce409-d525-4ecc-b3b5-04d3a9b55ee0 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:23.304809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cd1ce409-d525-4ecc-b3b5-04d3a9b55ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:23.306599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.329345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 842/1703] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:23.332497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ce409-d525-4ecc-b3b5-04d3a9b55ee0 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.334111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cd1ce409-d525-4ecc-b3b5-04d3a9b55ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:23.360215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ce409-d525-4ecc-b3b5-04d3a9b55ee0 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.370654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c425b8-9511-4533-a440-e5ad6dc14dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.371279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c425b8-9511-4533-a440-e5ad6dc14dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.371556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c425b8-9511-4533-a440-e5ad6dc14dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.371882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c425b8-9511-4533-a440-e5ad6dc14dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.372579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c425b8-9511-4533-a440-e5ad6dc14dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.374596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c425b8-9511-4533-a440-e5ad6dc14dc2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.406299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 843/1704] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:23.408152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97268c-cc7f-4692-ad8e-77b394d358e8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.408498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97268c-cc7f-4692-ad8e-77b394d358e8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.409277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97268c-cc7f-4692-ad8e-77b394d358e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.488768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-5e97268c-cc7f-4692-ad8e-77b394d358e8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:23.492665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5e97268c-cc7f-4692-ad8e-77b394d358e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:23.516290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 844/1705] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:23.517545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97268c-cc7f-4692-ad8e-77b394d358e8 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.551711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97268c-cc7f-4692-ad8e-77b394d358e8 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.561766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62016dd039054c13a61cb63047970059'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.592686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.592686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.592686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.592686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.593636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:23.593636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.593636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.602195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62016dd039054c13a61cb63047970059'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.645056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 845/1706] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:23.646062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.691887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bdc6e-417d-4c93-865e-e92ad4c1a216 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.701697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62016dd039054c13a61cb63047970059'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.702485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62016dd039054c13a61cb63047970059'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.704377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00d05075-9159-43d3-a7b8-cbdfcccdae6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62016dd039054c13a61cb63047970059'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:23.707093 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.707705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.707981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.710110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] PATH_INFO: `/v3/projects/0b19b0550c95483593f5e79669e881cc/users/5b28a18a4aee4e298dd6dabb34938bd9/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.739777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 862/1707] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:23 2022] DELETE /identity/v3/users/62016dd039054c13a61cb63047970059 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:23.748302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efdd3c-9c6f-438c-8ef6-9df497cc7f80 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.755962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efdd3c-9c6f-438c-8ef6-9df497cc7f80 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.756685 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efdd3c-9c6f-438c-8ef6-9df497cc7f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.760458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b19b0550c95483593f5e79669e881cc, user_id=5b28a18a4aee4e298dd6dabb34938bd9, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.766385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.800105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-52efdd3c-9c6f-438c-8ef6-9df497cc7f80 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:23.802183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-52efdd3c-9c6f-438c-8ef6-9df497cc7f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:23.899824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 846/1708] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:23 2022] PUT /identity/v3/projects/0b19b0550c95483593f5e79669e881cc/users/5b28a18a4aee4e298dd6dabb34938bd9/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:23.900918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.905222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:23.946626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 863/1709] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:23.950733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efdd3c-9c6f-438c-8ef6-9df497cc7f80 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.954485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-52efdd3c-9c6f-438c-8ef6-9df497cc7f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:23.976269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efdd3c-9c6f-438c-8ef6-9df497cc7f80 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:24.084182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a45b59-2997-49c9-9b92-8fdf4121c8fa admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:24.098533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5452d0-cdfd-4e49-bc2a-6476bde0ae46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:24.099834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5452d0-cdfd-4e49-bc2a-6476bde0ae46 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.100489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5452d0-cdfd-4e49-bc2a-6476bde0ae46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.109677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5452d0-cdfd-4e49-bc2a-6476bde0ae46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.109677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5452d0-cdfd-4e49-bc2a-6476bde0ae46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:24.109677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5452d0-cdfd-4e49-bc2a-6476bde0ae46 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:24.132578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20994a-902c-48cb-aca8-a2ff78348b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:24.133510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20994a-902c-48cb-aca8-a2ff78348b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.133818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20994a-902c-48cb-aca8-a2ff78348b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.134242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20994a-902c-48cb-aca8-a2ff78348b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.135079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20994a-902c-48cb-aca8-a2ff78348b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:24.137069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20994a-902c-48cb-aca8-a2ff78348b04 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:24.150036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 864/1710] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:24.151276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20994a-902c-48cb-aca8-a2ff78348b04 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:24.174993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20994a-902c-48cb-aca8-a2ff78348b04 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:24.182322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:24.182955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.183279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.183558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.184227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:24.184534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:24.188034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:24.238068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 865/1711] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:24.239150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:24.271321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f262c-5385-4584-879c-a658a5efa6e3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:24.277846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd377f-cf61-43b9-93c4-928d2d1c2324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:24.278706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd377f-cf61-43b9-93c4-928d2d1c2324 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.279161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd377f-cf61-43b9-93c4-928d2d1c2324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.279559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd377f-cf61-43b9-93c4-928d2d1c2324 admin admin] PATH_INFO: `/v3/projects/0b19b0550c95483593f5e79669e881cc/users/5b28a18a4aee4e298dd6dabb34938bd9/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.282222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd377f-cf61-43b9-93c4-928d2d1c2324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b19b0550c95483593f5e79669e881cc, user_id=5b28a18a4aee4e298dd6dabb34938bd9, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:24.286846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd377f-cf61-43b9-93c4-928d2d1c2324 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:24.315523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 866/1712] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:24 2022] PUT /identity/v3/projects/0b19b0550c95483593f5e79669e881cc/users/5b28a18a4aee4e298dd6dabb34938bd9/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:24.315523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd377f-cf61-43b9-93c4-928d2d1c2324 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:24.325922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 847/1713] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:23 2022] POST /identity/v3/projects => generated 377 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:24.327362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2194e1-9c92-4b8e-9fe7-2a47ced9f5b7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.330164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2194e1-9c92-4b8e-9fe7-2a47ced9f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.330164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2194e1-9c92-4b8e-9fe7-2a47ced9f5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.385342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:24.515430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-1e2194e1-9c92-4b8e-9fe7-2a47ced9f5b7 None None] MFA Rules not processed for user `5b28a18a4aee4e298dd6dabb34938bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:24.517941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1e2194e1-9c92-4b8e-9fe7-2a47ced9f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:24.634548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd377f-cf61-43b9-93c4-928d2d1c2324 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:24.764590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 848/1714] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:24.770080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c99ef2-7461-47b8-a925-37250f804ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:24.770738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99ef2-7461-47b8-a925-37250f804ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.771025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99ef2-7461-47b8-a925-37250f804ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.771516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99ef2-7461-47b8-a925-37250f804ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.771987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c99ef2-7461-47b8-a925-37250f804ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:24.774921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c99ef2-7461-47b8-a925-37250f804ab7 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:24.776647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2194e1-9c92-4b8e-9fe7-2a47ced9f5b7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:24.826177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2194e1-9c92-4b8e-9fe7-2a47ced9f5b7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:24.845194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:24.845878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.849462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.849749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.850512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:24.860413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:24.923079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 867/1715] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:24 2022] POST /identity/v3/projects => generated 373 bytes in 615 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:24.924549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85e49-4385-4912-a1a4-37a7e835d182 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.924845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85e49-4385-4912-a1a4-37a7e835d182 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.925124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85e49-4385-4912-a1a4-37a7e835d182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.990102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-83e85e49-4385-4912-a1a4-37a7e835d182 None None] MFA Rules not processed for user `612dfebb8ebc401bb81d3d3333577566`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:24.991392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-83e85e49-4385-4912-a1a4-37a7e835d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:25.035377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 849/1716] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:24 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:25.036343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:25.042725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:25.128221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 868/1717] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:25.130014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e85e49-4385-4912-a1a4-37a7e835d182 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:25.137615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-83e85e49-4385-4912-a1a4-37a7e835d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:25.222728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61205af-a391-47db-9fb6-5a5f83fd4ead admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:25.280190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:25.292119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e85e49-4385-4912-a1a4-37a7e835d182 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:25.319926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:25.320959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:25.321553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:25.322022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] PATH_INFO: `/v3/projects/159adfdbdf05482ca799fa76f4814ee3` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:25.323784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=159adfdbdf05482ca799fa76f4814ee3)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:25.326766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:25.390000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8452d-58da-4635-9e62-bdbe5d5781e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:25.394334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8452d-58da-4635-9e62-bdbe5d5781e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:25.395688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8452d-58da-4635-9e62-bdbe5d5781e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:25.398521 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8452d-58da-4635-9e62-bdbe5d5781e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:25.494853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8452d-58da-4635-9e62-bdbe5d5781e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:25.496397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8452d-58da-4635-9e62-bdbe5d5781e5 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:25.509418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '159adfdbdf05482ca799fa76f4814ee3', 'request_id': 'req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:25.512918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 850/1718] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:25.514230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8452d-58da-4635-9e62-bdbe5d5781e5 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:25.533212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8452d-58da-4635-9e62-bdbe5d5781e5 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:25.623659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:25.624293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:25.624571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:25.624896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:25.625543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:25.628433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:25.645312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:25.673371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '159adfdbdf05482ca799fa76f4814ee3', 'request_id': 'req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:25.684724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '159adfdbdf05482ca799fa76f4814ee3', 'request_id': 'req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:25.710334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] The token cache is being invalidate because project 159adfdbdf05482ca799fa76f4814ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:25.710686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:25.712743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-da22f3d0-46b9-4eee-ac57-2b34a77b59cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:25.724021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 869/1719] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:25 2022] DELETE /identity/v3/projects/159adfdbdf05482ca799fa76f4814ee3 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:25.725022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78276099-673f-4140-a334-a049bb6121ae None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:25.725310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78276099-673f-4140-a334-a049bb6121ae None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:25.725585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78276099-673f-4140-a334-a049bb6121ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:25.756103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-78276099-673f-4140-a334-a049bb6121ae None None] MFA Rules not processed for user `251bc85f89774142a97acd77895b14e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:25.759132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-78276099-673f-4140-a334-a049bb6121ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:25.802783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 851/1720] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:25 2022] POST /identity/v3/users => generated 287 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:25.805488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:25.809430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:25.836914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea747d-0e45-4b50-b554-2a6235575ea6 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:25.852694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20549a6b-1690-436f-ae1a-66bcf5b2ecbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:25.853645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549a6b-1690-436f-ae1a-66bcf5b2ecbc admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:25.854788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549a6b-1690-436f-ae1a-66bcf5b2ecbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:25.856129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549a6b-1690-436f-ae1a-66bcf5b2ecbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:25.857447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20549a6b-1690-436f-ae1a-66bcf5b2ecbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:25.864712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 870/1721] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:25.865829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78276099-673f-4140-a334-a049bb6121ae None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:25.867470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-78276099-673f-4140-a334-a049bb6121ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:25.879273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20549a6b-1690-436f-ae1a-66bcf5b2ecbc admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:25.914015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 852/1722] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:25.915438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4faf2-29df-4aec-b0b6-0aebd5fbb643 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:25.915729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4faf2-29df-4aec-b0b6-0aebd5fbb643 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:25.916005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4faf2-29df-4aec-b0b6-0aebd5fbb643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:25.985978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ccb4faf2-29df-4aec-b0b6-0aebd5fbb643 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:25.988478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ccb4faf2-29df-4aec-b0b6-0aebd5fbb643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:25.997812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 853/1723] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:25.998912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4faf2-29df-4aec-b0b6-0aebd5fbb643 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.000741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ccb4faf2-29df-4aec-b0b6-0aebd5fbb643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:26.004403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78276099-673f-4140-a334-a049bb6121ae None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:26.010711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223f0f3-26b5-4a19-aded-22e17bfb5e1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:26.012191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223f0f3-26b5-4a19-aded-22e17bfb5e1d service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.012592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223f0f3-26b5-4a19-aded-22e17bfb5e1d service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.013019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223f0f3-26b5-4a19-aded-22e17bfb5e1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.015442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b223f0f3-26b5-4a19-aded-22e17bfb5e1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:26.016861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4faf2-29df-4aec-b0b6-0aebd5fbb643 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:26.024272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c08f0-c269-4477-af76-1dc3833696bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:26.025058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c08f0-c269-4477-af76-1dc3833696bf admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.028104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c08f0-c269-4477-af76-1dc3833696bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.028270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c08f0-c269-4477-af76-1dc3833696bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.029081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c08f0-c269-4477-af76-1dc3833696bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:26.030828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c08f0-c269-4477-af76-1dc3833696bf admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:26.053224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 854/1724] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:26.059518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c08f0-c269-4477-af76-1dc3833696bf admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.072846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c08f0-c269-4477-af76-1dc3833696bf admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:26.088907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:26.089534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.090175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.090464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.091123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:26.092756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:26.124884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223f0f3-26b5-4a19-aded-22e17bfb5e1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:26.125508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 855/1725] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:26.125508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.126981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223f0f3-26b5-4a19-aded-22e17bfb5e1d service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:26.129248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:26.145368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 871/1726] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:26.147992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a348a6-ed32-46c9-88a7-a74c3c6762ff None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.148650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a348a6-ed32-46c9-88a7-a74c3c6762ff None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.149092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a348a6-ed32-46c9-88a7-a74c3c6762ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.186414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-22a348a6-ed32-46c9-88a7-a74c3c6762ff None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:26.187831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-22a348a6-ed32-46c9-88a7-a74c3c6762ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:26.207752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 872/1727] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:26.207752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a348a6-ed32-46c9-88a7-a74c3c6762ff None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.219485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a348a6-ed32-46c9-88a7-a74c3c6762ff None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:26.226698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d81e3-2fb2-4697-b6ab-a9eb8c1a9fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:26.227548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d81e3-2fb2-4697-b6ab-a9eb8c1a9fda admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.228524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d81e3-2fb2-4697-b6ab-a9eb8c1a9fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.228524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d81e3-2fb2-4697-b6ab-a9eb8c1a9fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.229004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d81e3-2fb2-4697-b6ab-a9eb8c1a9fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:26.231282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d81e3-2fb2-4697-b6ab-a9eb8c1a9fda admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:26.267412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 873/1728] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:26.267412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d81e3-2fb2-4697-b6ab-a9eb8c1a9fda admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.281713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d81e3-2fb2-4697-b6ab-a9eb8c1a9fda admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:26.292209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29f284-248b-4c91-929e-6a6209b78794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:26.293053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29f284-248b-4c91-929e-6a6209b78794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.293443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29f284-248b-4c91-929e-6a6209b78794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.293887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29f284-248b-4c91-929e-6a6209b78794 admin admin] PATH_INFO: `/v3/projects/55eab4958382416681cd76e5a6a9b0c0/users/c403594e98cf4dcf950ac13172d99f53/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.303693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f150bf-dafd-44bc-9716-d388c4e631a7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:26.309712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:26.310785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.311384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.312739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.319154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29f284-248b-4c91-929e-6a6209b78794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55eab4958382416681cd76e5a6a9b0c0, user_id=c403594e98cf4dcf950ac13172d99f53, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:26.319273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:26.319925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29f284-248b-4c91-929e-6a6209b78794 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:26.451441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 874/1729] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:26 2022] PUT /identity/v3/projects/55eab4958382416681cd76e5a6a9b0c0/users/c403594e98cf4dcf950ac13172d99f53/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:26.452998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29f284-248b-4c91-929e-6a6209b78794 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.473623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:26.475948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:26.504408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 856/1730] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:26.506382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.534529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3de417-a6b0-461d-a6b3-2781e6cf35eb service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:26.663778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29f284-248b-4c91-929e-6a6209b78794 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:26.669796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f00802-ee9d-4f62-b358-bed80141ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:26.670919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00802-ee9d-4f62-b358-bed80141ac5b admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.671316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00802-ee9d-4f62-b358-bed80141ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.671699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00802-ee9d-4f62-b358-bed80141ac5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.672655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f00802-ee9d-4f62-b358-bed80141ac5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:26.673082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f00802-ee9d-4f62-b358-bed80141ac5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:26.675561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f00802-ee9d-4f62-b358-bed80141ac5b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:26.700193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 875/1731] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:26.702233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f05254-c3d0-4bde-bea6-085be550b8bb None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.702643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f05254-c3d0-4bde-bea6-085be550b8bb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.703081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f05254-c3d0-4bde-bea6-085be550b8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.737702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28182ce-d327-4b3e-bf0b-8840857fd5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:26.740995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28182ce-d327-4b3e-bf0b-8840857fd5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:26.741290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28182ce-d327-4b3e-bf0b-8840857fd5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:26.745752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28182ce-d327-4b3e-bf0b-8840857fd5ad admin admin] PATH_INFO: `/v3/projects/d614a23b1f164a67a833715689b2c1e4/users/5e46a6823b28422fa176c73cd04cff4e/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:26.755700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-70f05254-c3d0-4bde-bea6-085be550b8bb None None] MFA Rules not processed for user `c403594e98cf4dcf950ac13172d99f53`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:26.760287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-70f05254-c3d0-4bde-bea6-085be550b8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:26.795622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28182ce-d327-4b3e-bf0b-8840857fd5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d614a23b1f164a67a833715689b2c1e4, user_id=5e46a6823b28422fa176c73cd04cff4e, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:26.802312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28182ce-d327-4b3e-bf0b-8840857fd5ad admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:26.917828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 876/1732] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:26.918932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f05254-c3d0-4bde-bea6-085be550b8bb None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.950183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 857/1733] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:26 2022] PUT /identity/v3/projects/d614a23b1f164a67a833715689b2c1e4/users/5e46a6823b28422fa176c73cd04cff4e/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:26.952108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28182ce-d327-4b3e-bf0b-8840857fd5ad admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:26.952571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f05254-c3d0-4bde-bea6-085be550b8bb None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:27.084562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:27.098705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c122736-01a7-4a30-8566-ffd2201eb01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:27.100238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c122736-01a7-4a30-8566-ffd2201eb01d admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:27.100668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c122736-01a7-4a30-8566-ffd2201eb01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:27.102240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c122736-01a7-4a30-8566-ffd2201eb01d admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:27.102925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c122736-01a7-4a30-8566-ffd2201eb01d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:27.111912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c122736-01a7-4a30-8566-ffd2201eb01d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:27.125312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28182ce-d327-4b3e-bf0b-8840857fd5ad admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:27.142267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f34c1-2e40-4c5c-ab25-b74fe86ffbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:27.142906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f34c1-2e40-4c5c-ab25-b74fe86ffbfd admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:27.143990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f34c1-2e40-4c5c-ab25-b74fe86ffbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:27.144281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f34c1-2e40-4c5c-ab25-b74fe86ffbfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:27.144923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f34c1-2e40-4c5c-ab25-b74fe86ffbfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:27.147042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f34c1-2e40-4c5c-ab25-b74fe86ffbfd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:27.282127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 877/1734] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:26 2022] POST /identity/v3/projects => generated 365 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:27.291159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 858/1735] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:26 2022] POST /identity/v3/projects => generated 377 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:27.294412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa21c1-c213-454b-a21e-f3834fc78a4b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:27.294861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa21c1-c213-454b-a21e-f3834fc78a4b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:27.295689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa21c1-c213-454b-a21e-f3834fc78a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:27.298127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5eb58-c99c-4712-8ab8-4ed913da70f7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:27.298127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5eb58-c99c-4712-8ab8-4ed913da70f7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:27.298127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5eb58-c99c-4712-8ab8-4ed913da70f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:27.396866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d2aa21c1-c213-454b-a21e-f3834fc78a4b None None] MFA Rules not processed for user `5e46a6823b28422fa176c73cd04cff4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:27.399104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d2aa21c1-c213-454b-a21e-f3834fc78a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:27.411552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-8df5eb58-c99c-4712-8ab8-4ed913da70f7 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:27.412841 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8df5eb58-c99c-4712-8ab8-4ed913da70f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:27.433610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:27.574813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 878/1736] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:27.578262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa21c1-c213-454b-a21e-f3834fc78a4b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:27.618492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 859/1737] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:49:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:27.619898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5eb58-c99c-4712-8ab8-4ed913da70f7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:27.726112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5eb58-c99c-4712-8ab8-4ed913da70f7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:27.731068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274fdac-a600-4e33-8ec8-933bf5db2729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:27.731773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274fdac-a600-4e33-8ec8-933bf5db2729 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:27.732359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274fdac-a600-4e33-8ec8-933bf5db2729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:27.732760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274fdac-a600-4e33-8ec8-933bf5db2729 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:27.733508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274fdac-a600-4e33-8ec8-933bf5db2729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:27.736331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274fdac-a600-4e33-8ec8-933bf5db2729 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:27.750179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa21c1-c213-454b-a21e-f3834fc78a4b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:27.761300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d20c-d90f-4fd9-a51a-eef45c6947d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:27.762084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d20c-d90f-4fd9-a51a-eef45c6947d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:27.762525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d20c-d90f-4fd9-a51a-eef45c6947d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:27.763422 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d20c-d90f-4fd9-a51a-eef45c6947d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:27.768758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530d20c-d90f-4fd9-a51a-eef45c6947d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:27.768758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530d20c-d90f-4fd9-a51a-eef45c6947d2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:27.886232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 879/1738] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:27 2022] POST /identity/v3/users => generated 295 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:27.891274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d20c-d90f-4fd9-a51a-eef45c6947d2 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:27.891505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 860/1739] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:27 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:27.892685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274fdac-a600-4e33-8ec8-933bf5db2729 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:27.924396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274fdac-a600-4e33-8ec8-933bf5db2729 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:27.925079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d20c-d90f-4fd9-a51a-eef45c6947d2 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:27.935701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415e998-2cba-49e2-8cd4-581ac90e1cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:27.938953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ed0bd-c6a1-4d54-97b1-1667aad6a8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:27.938953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed0bd-c6a1-4d54-97b1-1667aad6a8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:27.938953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed0bd-c6a1-4d54-97b1-1667aad6a8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:27.938953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed0bd-c6a1-4d54-97b1-1667aad6a8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:27.939373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ed0bd-c6a1-4d54-97b1-1667aad6a8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:27.940814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415e998-2cba-49e2-8cd4-581ac90e1cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:27.941264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415e998-2cba-49e2-8cd4-581ac90e1cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:27.941658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415e998-2cba-49e2-8cd4-581ac90e1cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:27.945380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415e998-2cba-49e2-8cd4-581ac90e1cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:27.945380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415e998-2cba-49e2-8cd4-581ac90e1cc9 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:27.946792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ed0bd-c6a1-4d54-97b1-1667aad6a8d6 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:27.975929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 861/1740] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:27.977206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ed0bd-c6a1-4d54-97b1-1667aad6a8d6 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:27.994001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ed0bd-c6a1-4d54-97b1-1667aad6a8d6 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.002079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:28.006204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ab74e-a36e-4862-bbce-df3badd5aa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:28.007147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ab74e-a36e-4862-bbce-df3badd5aa65 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.007681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ab74e-a36e-4862-bbce-df3badd5aa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.008454 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ab74e-a36e-4862-bbce-df3badd5aa65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.019649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ab74e-a36e-4862-bbce-df3badd5aa65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:28.024677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ab74e-a36e-4862-bbce-df3badd5aa65 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:28.071886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 880/1741] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:27 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:28.071886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415e998-2cba-49e2-8cd4-581ac90e1cc9 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:28.083456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 862/1742] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:28.087695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2af24-8b3f-4098-93cb-4db3770db255 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.089402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2af24-8b3f-4098-93cb-4db3770db255 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.090935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2af24-8b3f-4098-93cb-4db3770db255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.145003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-81e2af24-8b3f-4098-93cb-4db3770db255 None None] MFA Rules not processed for user `2b6b0d0cbdb54a46a7e9d6936c47cc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:28.146299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-81e2af24-8b3f-4098-93cb-4db3770db255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:28.186151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415e998-2cba-49e2-8cd4-581ac90e1cc9 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.206457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4802a7-2b70-417d-883f-feba63af474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:28.207346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4802a7-2b70-417d-883f-feba63af474b admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.207740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4802a7-2b70-417d-883f-feba63af474b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.208131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4802a7-2b70-417d-883f-feba63af474b admin admin] PATH_INFO: `/v3/projects/8678c6cc8897418fb26532ec74186e15/users/01fe0b3cf1384a1a86b30ea41a5e15a4/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.242567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4802a7-2b70-417d-883f-feba63af474b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8678c6cc8897418fb26532ec74186e15, user_id=01fe0b3cf1384a1a86b30ea41a5e15a4, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:28.245575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4802a7-2b70-417d-883f-feba63af474b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:28.270618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 863/1743] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:28.270618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2af24-8b3f-4098-93cb-4db3770db255 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:28.304069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2af24-8b3f-4098-93cb-4db3770db255 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.310449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:28.311176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.311530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.311873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.312600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:28.317075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:28.417202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 881/1744] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:28 2022] PUT /identity/v3/projects/8678c6cc8897418fb26532ec74186e15/users/01fe0b3cf1384a1a86b30ea41a5e15a4/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:28.417202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4802a7-2b70-417d-883f-feba63af474b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:28.474638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 864/1745] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:28 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:28.476100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:28.480508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:28.586173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4802a7-2b70-417d-883f-feba63af474b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.598104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9179b-5e41-426e-b73d-7eeb3b02b61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:28.599947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9179b-5e41-426e-b73d-7eeb3b02b61f admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.600516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9179b-5e41-426e-b73d-7eeb3b02b61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.601030 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9179b-5e41-426e-b73d-7eeb3b02b61f admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.602497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9179b-5e41-426e-b73d-7eeb3b02b61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:28.605210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9179b-5e41-426e-b73d-7eeb3b02b61f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:28.629305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 882/1746] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:28.630972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9179b-5e41-426e-b73d-7eeb3b02b61f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:28.649795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:28.657682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9179b-5e41-426e-b73d-7eeb3b02b61f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.676765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59f0aa-3997-4d17-a0e4-7e8c79a007d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:28.677762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f0aa-3997-4d17-a0e4-7e8c79a007d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.678255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f0aa-3997-4d17-a0e4-7e8c79a007d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.679906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f0aa-3997-4d17-a0e4-7e8c79a007d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.680883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59f0aa-3997-4d17-a0e4-7e8c79a007d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:28.683006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59f0aa-3997-4d17-a0e4-7e8c79a007d0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:28.708741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 883/1747] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:28.711278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59f0aa-3997-4d17-a0e4-7e8c79a007d0 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:28.735099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59f0aa-3997-4d17-a0e4-7e8c79a007d0 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.735642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaba3c1-55fe-4cb0-a78d-15603b216a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:28.736398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaba3c1-55fe-4cb0-a78d-15603b216a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.737479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaba3c1-55fe-4cb0-a78d-15603b216a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.737881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaba3c1-55fe-4cb0-a78d-15603b216a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.739434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaba3c1-55fe-4cb0-a78d-15603b216a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:28.741699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaba3c1-55fe-4cb0-a78d-15603b216a3c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:28.761622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 884/1748] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:28.761622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaba3c1-55fe-4cb0-a78d-15603b216a3c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:28.774083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaba3c1-55fe-4cb0-a78d-15603b216a3c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.782007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c507885-1f97-4495-99fa-45f4016a4966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:28.783546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507885-1f97-4495-99fa-45f4016a4966 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.783546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507885-1f97-4495-99fa-45f4016a4966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.783754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507885-1f97-4495-99fa-45f4016a4966 admin admin] PATH_INFO: `/v3/projects/ff30b77ae3444cba825217f473f05920/users/1098467b5e32455397c6f187bdde6838/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.810502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c507885-1f97-4495-99fa-45f4016a4966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff30b77ae3444cba825217f473f05920, user_id=1098467b5e32455397c6f187bdde6838, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:28.825977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c507885-1f97-4495-99fa-45f4016a4966 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:28.886510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc7994-cb61-4ad5-8f78-f33fb135a050 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.948579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 885/1749] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:28 2022] PUT /identity/v3/projects/ff30b77ae3444cba825217f473f05920/users/1098467b5e32455397c6f187bdde6838/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:28.950573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c507885-1f97-4495-99fa-45f4016a4966 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:29.118454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c507885-1f97-4495-99fa-45f4016a4966 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:29.133608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76be721-1399-4c4d-9ae4-536c8ffef5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:29.135103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be721-1399-4c4d-9ae4-536c8ffef5c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.135377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be721-1399-4c4d-9ae4-536c8ffef5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.136238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be721-1399-4c4d-9ae4-536c8ffef5c7 admin admin] PATH_INFO: `/v3/projects/8678c6cc8897418fb26532ec74186e15/users/01fe0b3cf1384a1a86b30ea41a5e15a4/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.144459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76be721-1399-4c4d-9ae4-536c8ffef5c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8678c6cc8897418fb26532ec74186e15, user_id=01fe0b3cf1384a1a86b30ea41a5e15a4, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:29.146272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212534e0-4165-4634-b3ab-0cc00c89086a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:29.146914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212534e0-4165-4634-b3ab-0cc00c89086a service swift] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.147201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212534e0-4165-4634-b3ab-0cc00c89086a service swift] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.147853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212534e0-4165-4634-b3ab-0cc00c89086a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.154963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76be721-1399-4c4d-9ae4-536c8ffef5c7 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:29.198086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 886/1750] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:28 2022] PUT /identity/v3/projects/8678c6cc8897418fb26532ec74186e15/users/01fe0b3cf1384a1a86b30ea41a5e15a4/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:29.199119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76be721-1399-4c4d-9ae4-536c8ffef5c7 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:29.255473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212534e0-4165-4634-b3ab-0cc00c89086a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:29.258397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212534e0-4165-4634-b3ab-0cc00c89086a service swift] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:29.275580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 865/1751] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:49:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 800 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:49:29.279961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b46851-3d19-4d88-b527-395c3cad0327 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.282099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b46851-3d19-4d88-b527-395c3cad0327 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.282887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b46851-3d19-4d88-b527-395c3cad0327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.313143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76be721-1399-4c4d-9ae4-536c8ffef5c7 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:29.330230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b0b46851-3d19-4d88-b527-395c3cad0327 None None] MFA Rules not processed for user `1098467b5e32455397c6f187bdde6838`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:29.333383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b0b46851-3d19-4d88-b527-395c3cad0327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:29.335909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dc30b-3a4f-4c4f-9041-ae1312208293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:29.336876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dc30b-3a4f-4c4f-9041-ae1312208293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.336988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dc30b-3a4f-4c4f-9041-ae1312208293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.337235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dc30b-3a4f-4c4f-9041-ae1312208293 admin admin] PATH_INFO: `/v3/projects/8f58c3168aca475d8555ea1548800b52/users/f44f8fe20122489cb405f551465f8b2a/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.442260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dc30b-3a4f-4c4f-9041-ae1312208293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f58c3168aca475d8555ea1548800b52, user_id=f44f8fe20122489cb405f551465f8b2a, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:29.449743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dc30b-3a4f-4c4f-9041-ae1312208293 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:29.550706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 887/1752] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:29 2022] PUT /identity/v3/projects/8f58c3168aca475d8555ea1548800b52/users/f44f8fe20122489cb405f551465f8b2a/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:29.554198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b6c57-faa2-46f3-ba5c-ea2c517f92f2 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.554481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b6c57-faa2-46f3-ba5c-ea2c517f92f2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.554855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b6c57-faa2-46f3-ba5c-ea2c517f92f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.593733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:29.604464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 866/1753] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:29.609683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a166aa-0bad-43fb-bf22-0dfb3f557322 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.610095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a166aa-0bad-43fb-bf22-0dfb3f557322 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.610265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a166aa-0bad-43fb-bf22-0dfb3f557322 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.618290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 867/1754] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 24 17:49:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:29.630781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a166aa-0bad-43fb-bf22-0dfb3f557322 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:29.699390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-654b6c57-faa2-46f3-ba5c-ea2c517f92f2 None None] MFA Rules not processed for user `01fe0b3cf1384a1a86b30ea41a5e15a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:29.700715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-654b6c57-faa2-46f3-ba5c-ea2c517f92f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:29.809974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a166aa-0bad-43fb-bf22-0dfb3f557322 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:29.820732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0be81-2d83-421f-a739-003bcc6b1342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:29.821497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0be81-2d83-421f-a739-003bcc6b1342 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.821869 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0be81-2d83-421f-a739-003bcc6b1342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.822566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0be81-2d83-421f-a739-003bcc6b1342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.823727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0be81-2d83-421f-a739-003bcc6b1342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:29.825755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0be81-2d83-421f-a739-003bcc6b1342 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:29.886377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 868/1755] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:29.887518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0be81-2d83-421f-a739-003bcc6b1342 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:29.911209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 888/1756] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:29.913013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baea6d-ee87-4954-b148-3738996a1048 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.913534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baea6d-ee87-4954-b148-3738996a1048 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.914210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baea6d-ee87-4954-b148-3738996a1048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.944540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0be81-2d83-421f-a739-003bcc6b1342 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:29.948409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-46baea6d-ee87-4954-b148-3738996a1048 None None] MFA Rules not processed for user `efc068011c14492d9ba483650cf632e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:29.950042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-46baea6d-ee87-4954-b148-3738996a1048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:29.961246 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbf8da-15af-4747-b73a-a0fc594615f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:29.962531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbf8da-15af-4747-b73a-a0fc594615f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.963213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbf8da-15af-4747-b73a-a0fc594615f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.963826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbf8da-15af-4747-b73a-a0fc594615f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.964919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbf8da-15af-4747-b73a-a0fc594615f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:29.970153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbf8da-15af-4747-b73a-a0fc594615f1 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:30.109125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 869/1757] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:29 2022] POST /identity/v3/projects => generated 361 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:30.121401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb211a6-cf70-46f8-bfc5-583f756456e1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.121740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb211a6-cf70-46f8-bfc5-583f756456e1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.122013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb211a6-cf70-46f8-bfc5-583f756456e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.133910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 889/1758] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:49:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:30.136808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baea6d-ee87-4954-b148-3738996a1048 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.177032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baea6d-ee87-4954-b148-3738996a1048 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.193118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9eb211a6-cf70-46f8-bfc5-583f756456e1 None None] MFA Rules not processed for user `efc068011c14492d9ba483650cf632e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:30.194383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9eb211a6-cf70-46f8-bfc5-583f756456e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:30.386531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a42a8-15f2-477c-b5ef-fb6d1257b6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.389455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a42a8-15f2-477c-b5ef-fb6d1257b6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.392554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a42a8-15f2-477c-b5ef-fb6d1257b6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.393476 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a42a8-15f2-477c-b5ef-fb6d1257b6d4 admin admin] PATH_INFO: `/v3/projects/8f58c3168aca475d8555ea1548800b52/users/f44f8fe20122489cb405f551465f8b2a/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.393879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 870/1759] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Aug 24 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:30.395068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb211a6-cf70-46f8-bfc5-583f756456e1 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.404683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9eb211a6-cf70-46f8-bfc5-583f756456e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:30.405624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a42a8-15f2-477c-b5ef-fb6d1257b6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f58c3168aca475d8555ea1548800b52, user_id=f44f8fe20122489cb405f551465f8b2a, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:30.412236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a42a8-15f2-477c-b5ef-fb6d1257b6d4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:30.495719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 890/1760] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:30 2022] PUT /identity/v3/projects/8f58c3168aca475d8555ea1548800b52/users/f44f8fe20122489cb405f551465f8b2a/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:30.495719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a42a8-15f2-477c-b5ef-fb6d1257b6d4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.609293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a42a8-15f2-477c-b5ef-fb6d1257b6d4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.616009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.616792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.617415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.617577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.618354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:30.620588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:30.624148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb211a6-cf70-46f8-bfc5-583f756456e1 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.728252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 891/1761] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:30 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:30.730253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.735175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:30.758316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.760624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.762206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.763193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.770951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:31.097990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85190b93-1477-4023-a2cc-bb1be16a17ad admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:31.101287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.101287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.102059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ef6b7-edc2-427e-bddc-95fafb51cdd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:31.102059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef6b7-edc2-427e-bddc-95fafb51cdd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.102059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef6b7-edc2-427e-bddc-95fafb51cdd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.102059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef6b7-edc2-427e-bddc-95fafb51cdd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.102494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 871/1762] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:31.102494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:31.102494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:31.102877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef6b7-edc2-427e-bddc-95fafb51cdd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.102877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef6b7-edc2-427e-bddc-95fafb51cdd0 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.102877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 892/1763] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:31.103212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce2d06-7ee9-4d2f-b129-eb82d0f6d108 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:31.103308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc799f-976c-4142-967e-e1ea680ef2d9 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.103308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc799f-976c-4142-967e-e1ea680ef2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.103308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc799f-976c-4142-967e-e1ea680ef2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.103625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4811-826c-47fe-86a7-047b409ad756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:31.103625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4811-826c-47fe-86a7-047b409ad756 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.103625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4811-826c-47fe-86a7-047b409ad756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.103625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4811-826c-47fe-86a7-047b409ad756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.103987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f4811-826c-47fe-86a7-047b409ad756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.103987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f4811-826c-47fe-86a7-047b409ad756 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.103987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 872/1764] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:31.103987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4811-826c-47fe-86a7-047b409ad756 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:31.104346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-cefc799f-976c-4142-967e-e1ea680ef2d9 None None] MFA Rules not processed for user `f44f8fe20122489cb405f551465f8b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:31.104346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cefc799f-976c-4142-967e-e1ea680ef2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:31.104481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4811-826c-47fe-86a7-047b409ad756 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:31.104481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d35f0-922e-4db6-905a-5762f7cf3125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:31.104481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d35f0-922e-4db6-905a-5762f7cf3125 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.104481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d35f0-922e-4db6-905a-5762f7cf3125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.104784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d35f0-922e-4db6-905a-5762f7cf3125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.104784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d35f0-922e-4db6-905a-5762f7cf3125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.104784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d35f0-922e-4db6-905a-5762f7cf3125 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.104784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 873/1765] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:31.104784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d35f0-922e-4db6-905a-5762f7cf3125 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:31.104784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d35f0-922e-4db6-905a-5762f7cf3125 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:31.117626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c418a-2ca8-47dd-a7d6-861ab388836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:31.121263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c418a-2ca8-47dd-a7d6-861ab388836a admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.121263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c418a-2ca8-47dd-a7d6-861ab388836a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.121263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c418a-2ca8-47dd-a7d6-861ab388836a admin admin] PATH_INFO: `/v3/projects/2ad2f0cb69d74800ab0996ff41ad6319/users/0f4ef611bea649118a78f9da074de200/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.133892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 893/1766] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:31.158891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c418a-2ca8-47dd-a7d6-861ab388836a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad2f0cb69d74800ab0996ff41ad6319, user_id=0f4ef611bea649118a78f9da074de200, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.196539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c418a-2ca8-47dd-a7d6-861ab388836a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.407135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 874/1767] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:31 2022] PUT /identity/v3/projects/2ad2f0cb69d74800ab0996ff41ad6319/users/0f4ef611bea649118a78f9da074de200/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:31.413543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc799f-976c-4142-967e-e1ea680ef2d9 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:31.434317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc799f-976c-4142-967e-e1ea680ef2d9 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:31.437434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:31.672979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5084-5da4-4c41-91c6-6cc408d63cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:31.673613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5084-5da4-4c41-91c6-6cc408d63cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.677648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5084-5da4-4c41-91c6-6cc408d63cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.677648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5084-5da4-4c41-91c6-6cc408d63cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.681318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b5084-5da4-4c41-91c6-6cc408d63cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.685278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b5084-5da4-4c41-91c6-6cc408d63cc2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.744404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 894/1768] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:31.746425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c418a-2ca8-47dd-a7d6-861ab388836a admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:31.772218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c418a-2ca8-47dd-a7d6-861ab388836a admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:31.820806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77de1-9980-4abf-b59b-63b177408980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:31.823805 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77de1-9980-4abf-b59b-63b177408980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.831828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77de1-9980-4abf-b59b-63b177408980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.832657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77de1-9980-4abf-b59b-63b177408980 admin admin] PATH_INFO: `/v3/projects/2ad2f0cb69d74800ab0996ff41ad6319/users/0f4ef611bea649118a78f9da074de200/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.840481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c77de1-9980-4abf-b59b-63b177408980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad2f0cb69d74800ab0996ff41ad6319, user_id=0f4ef611bea649118a78f9da074de200, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.849532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c77de1-9980-4abf-b59b-63b177408980 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.906210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 875/1769] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:31 2022] PUT /identity/v3/projects/2ad2f0cb69d74800ab0996ff41ad6319/users/0f4ef611bea649118a78f9da074de200/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:31.906424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53721b94-bed3-48b9-bd86-264652b0e3dd None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.906424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53721b94-bed3-48b9-bd86-264652b0e3dd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.906424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53721b94-bed3-48b9-bd86-264652b0e3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.965177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-53721b94-bed3-48b9-bd86-264652b0e3dd None None] MFA Rules not processed for user `0f4ef611bea649118a78f9da074de200`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:31.972667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-53721b94-bed3-48b9-bd86-264652b0e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:32.146998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77de1-9980-4abf-b59b-63b177408980 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:32.204622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 895/1770] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:32.355986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53721b94-bed3-48b9-bd86-264652b0e3dd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:32.404955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77de1-9980-4abf-b59b-63b177408980 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:32.435218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481a860-557e-4ae0-810a-8aee32440992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:32.437754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481a860-557e-4ae0-810a-8aee32440992 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:32.438576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481a860-557e-4ae0-810a-8aee32440992 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:32.438876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481a860-557e-4ae0-810a-8aee32440992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:32.444270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:32.587191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53721b94-bed3-48b9-bd86-264652b0e3dd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:32.597811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40232e04-7ea9-49b1-90ac-b2afc4fd8421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:32.599081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232e04-7ea9-49b1-90ac-b2afc4fd8421 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:32.599366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232e04-7ea9-49b1-90ac-b2afc4fd8421 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:32.599638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232e04-7ea9-49b1-90ac-b2afc4fd8421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:32.657974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40232e04-7ea9-49b1-90ac-b2afc4fd8421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:32.659518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40232e04-7ea9-49b1-90ac-b2afc4fd8421 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:32.665485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481a860-557e-4ae0-810a-8aee32440992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:32.675352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481a860-557e-4ae0-810a-8aee32440992 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:32.696349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 896/1771] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:32.699178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 876/1772] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:32.985223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe178-4701-4fda-9804-b080e9671ab1 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:32.985755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe178-4701-4fda-9804-b080e9671ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:32.986065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe178-4701-4fda-9804-b080e9671ab1 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:32.989551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 897/1773] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:49:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:33.002432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:33.007365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672d832-f941-4aa3-8d50-a9c2a70725cf None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.007840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672d832-f941-4aa3-8d50-a9c2a70725cf None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.008215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672d832-f941-4aa3-8d50-a9c2a70725cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.047667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9672d832-f941-4aa3-8d50-a9c2a70725cf None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:33.050440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9672d832-f941-4aa3-8d50-a9c2a70725cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:33.116004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fe178-4701-4fda-9804-b080e9671ab1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.144684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fe178-4701-4fda-9804-b080e9671ab1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.157260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7e617-3b7c-40c1-911f-ffd2b0ff842d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.157993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7e617-3b7c-40c1-911f-ffd2b0ff842d service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.158344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7e617-3b7c-40c1-911f-ffd2b0ff842d service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.158875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7e617-3b7c-40c1-911f-ffd2b0ff842d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.318123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7e617-3b7c-40c1-911f-ffd2b0ff842d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.321677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7e617-3b7c-40c1-911f-ffd2b0ff842d service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.328167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 877/1774] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:49:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:33.340451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 898/1775] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:33.344164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672d832-f941-4aa3-8d50-a9c2a70725cf None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.345830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9672d832-f941-4aa3-8d50-a9c2a70725cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:33.367370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672d832-f941-4aa3-8d50-a9c2a70725cf None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.376575 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff811098-f704-47ab-aa01-fbe2645e1770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.381459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff811098-f704-47ab-aa01-fbe2645e1770 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.381459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff811098-f704-47ab-aa01-fbe2645e1770 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.381459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff811098-f704-47ab-aa01-fbe2645e1770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.388714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff811098-f704-47ab-aa01-fbe2645e1770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.390517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff811098-f704-47ab-aa01-fbe2645e1770 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.404117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 878/1776] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:33.982816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:34.091417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:34.113491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4279c1-e170-4775-b737-d4d3cf20ebad None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:34.114128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4279c1-e170-4775-b737-d4d3cf20ebad None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:34.114690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4279c1-e170-4775-b737-d4d3cf20ebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:34.138655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-bb4279c1-e170-4775-b737-d4d3cf20ebad None None] MFA Rules not processed for user `612dfebb8ebc401bb81d3d3333577566`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:34.140151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bb4279c1-e170-4775-b737-d4d3cf20ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:34.283142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 899/1777] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:34.376118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff811098-f704-47ab-aa01-fbe2645e1770 service nova] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:34.528208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff811098-f704-47ab-aa01-fbe2645e1770 service nova] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:34.545605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:34.548525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:34.548853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:34.551220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] PATH_INFO: `/v3/users/1098467b5e32455397c6f187bdde6838` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:34.553392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1098467b5e32455397c6f187bdde6838)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:34.569526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:34.868861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1098467b5e32455397c6f187bdde6838'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.917248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:34.942004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1098467b5e32455397c6f187bdde6838'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.945676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1098467b5e32455397c6f187bdde6838'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.953633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1098467b5e32455397c6f187bdde6838'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.954576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1098467b5e32455397c6f187bdde6838'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.995506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 879/1778] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:34 2022] DELETE /identity/v3/users/1098467b5e32455397c6f187bdde6838 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:34.999911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe58ce-95a4-46ac-9c72-9aecd2218591 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.000482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe58ce-95a4-46ac-9c72-9aecd2218591 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.000482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe58ce-95a4-46ac-9c72-9aecd2218591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.023293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1ebe58ce-95a4-46ac-9c72-9aecd2218591 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:35.024909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1ebe58ce-95a4-46ac-9c72-9aecd2218591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:35.114020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 900/1779] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:35.131023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.132543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:35.136410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:35.248041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5e4be-c0d2-41f8-9589-abb21689c710 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.251941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4fca1-f11a-4dd5-a4da-34a2741c65f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.252562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4fca1-f11a-4dd5-a4da-34a2741c65f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.252839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4fca1-f11a-4dd5-a4da-34a2741c65f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.253134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4fca1-f11a-4dd5-a4da-34a2741c65f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.263456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4fca1-f11a-4dd5-a4da-34a2741c65f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.265001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4fca1-f11a-4dd5-a4da-34a2741c65f2 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:35.299975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 880/1780] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:35.714434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:35.959976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:36.091152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe58ce-95a4-46ac-9c72-9aecd2218591 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:36.092646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1ebe58ce-95a4-46ac-9c72-9aecd2218591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:36.126129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe58ce-95a4-46ac-9c72-9aecd2218591 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:36.169968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:36.170816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.171622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.172751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] PATH_INFO: `/v3/projects/ff30b77ae3444cba825217f473f05920` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.176943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff30b77ae3444cba825217f473f05920)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:36.187745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.332764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff30b77ae3444cba825217f473f05920', 'request_id': 'req-2d57a23f-67fe-43ac-bb31-3d64d60c0358'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:36.501755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff30b77ae3444cba825217f473f05920', 'request_id': 'req-2d57a23f-67fe-43ac-bb31-3d64d60c0358'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:36.510081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff30b77ae3444cba825217f473f05920', 'request_id': 'req-2d57a23f-67fe-43ac-bb31-3d64d60c0358'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:36.524944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] The token cache is being invalidate because project ff30b77ae3444cba825217f473f05920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:36.525320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:36.526216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:36.528662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 901/1781] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:36 2022] DELETE /identity/v3/projects/ff30b77ae3444cba825217f473f05920 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:36.534228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4fca1-f11a-4dd5-a4da-34a2741c65f2 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:36.545682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4fca1-f11a-4dd5-a4da-34a2741c65f2 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:36.680248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:36.681471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.682057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.682638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] PATH_INFO: `/v3/users/0f4ef611bea649118a78f9da074de200` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.699448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4ef611bea649118a78f9da074de200)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:36.704304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.953335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4ef611bea649118a78f9da074de200'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.033911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4ef611bea649118a78f9da074de200'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.037957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4ef611bea649118a78f9da074de200'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.038784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:37.047478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4ef611bea649118a78f9da074de200'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.048734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d434168e-20b1-4451-a1f1-e7234fd5bdd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4ef611bea649118a78f9da074de200'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.062117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:37.116405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 881/1782] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:36 2022] DELETE /identity/v3/users/0f4ef611bea649118a78f9da074de200 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:37.548296 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:38.229868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.231764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:38.250611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57a23f-67fe-43ac-bb31-3d64d60c0358 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.319812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.320528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.320810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.321083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] PATH_INFO: `/v3/projects/2ad2f0cb69d74800ab0996ff41ad6319` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.322297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ad2f0cb69d74800ab0996ff41ad6319)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.324232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.326729 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:38.467189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ad2f0cb69d74800ab0996ff41ad6319', 'request_id': 'req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.536116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ad2f0cb69d74800ab0996ff41ad6319', 'request_id': 'req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.545843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ad2f0cb69d74800ab0996ff41ad6319', 'request_id': 'req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.558375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] The token cache is being invalidate because project 2ad2f0cb69d74800ab0996ff41ad6319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:38.562248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.563295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e2ab05c4-cc55-43f4-bfdd-c8292a8fc35f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.568927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 902/1783] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:38 2022] DELETE /identity/v3/projects/2ad2f0cb69d74800ab0996ff41ad6319 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:38.571616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b22e2-44fe-4e30-b2eb-8b4374f44623 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.572053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b22e2-44fe-4e30-b2eb-8b4374f44623 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.572306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b22e2-44fe-4e30-b2eb-8b4374f44623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.596261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-554b22e2-44fe-4e30-b2eb-8b4374f44623 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:38.599165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-554b22e2-44fe-4e30-b2eb-8b4374f44623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:38.659482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 882/1784] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:38.668659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c31c9-4f44-4e83-8a31-f72f714bd637 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.668948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c31c9-4f44-4e83-8a31-f72f714bd637 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.669222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c31c9-4f44-4e83-8a31-f72f714bd637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.689631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b70c31c9-4f44-4e83-8a31-f72f714bd637 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:38.690897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b70c31c9-4f44-4e83-8a31-f72f714bd637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:38.699049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 903/1785] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:38.703486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b22e2-44fe-4e30-b2eb-8b4374f44623 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.714041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b22e2-44fe-4e30-b2eb-8b4374f44623 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.718971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.719623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.719900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.720171 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.721426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:38.721738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.723470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.736086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 883/1786] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:38.741305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c31c9-4f44-4e83-8a31-f72f714bd637 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.750938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c31c9-4f44-4e83-8a31-f72f714bd637 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.756395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf879-e4e9-46ea-bca5-87a2746eb42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.756999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf879-e4e9-46ea-bca5-87a2746eb42e admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.757281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf879-e4e9-46ea-bca5-87a2746eb42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.757582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf879-e4e9-46ea-bca5-87a2746eb42e admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.758290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcf879-e4e9-46ea-bca5-87a2746eb42e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.760102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcf879-e4e9-46ea-bca5-87a2746eb42e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.847544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 904/1787] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:38 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:38.852871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.863262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a37dd4-f293-4896-9a63-43a799baa43c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.921303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a527f5-1fc0-4fb5-bc76-55ded6516c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.921950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a527f5-1fc0-4fb5-bc76-55ded6516c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.922236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a527f5-1fc0-4fb5-bc76-55ded6516c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.922520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a527f5-1fc0-4fb5-bc76-55ded6516c9e admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.923148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a527f5-1fc0-4fb5-bc76-55ded6516c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.925436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a527f5-1fc0-4fb5-bc76-55ded6516c9e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.022176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 884/1788] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:38 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:39.026232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf879-e4e9-46ea-bca5-87a2746eb42e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.038091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf879-e4e9-46ea-bca5-87a2746eb42e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.042814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837db104-51aa-4696-8503-95e1d513b3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.043426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db104-51aa-4696-8503-95e1d513b3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.043705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db104-51aa-4696-8503-95e1d513b3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.043980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db104-51aa-4696-8503-95e1d513b3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.044638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837db104-51aa-4696-8503-95e1d513b3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.046264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837db104-51aa-4696-8503-95e1d513b3ba admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.057750 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 905/1789] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:39.061925 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a527f5-1fc0-4fb5-bc76-55ded6516c9e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.072485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a527f5-1fc0-4fb5-bc76-55ded6516c9e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.077238 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e21b17-ab5b-4288-a338-0c2d1e677dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.077844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21b17-ab5b-4288-a338-0c2d1e677dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.078180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21b17-ab5b-4288-a338-0c2d1e677dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.078461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21b17-ab5b-4288-a338-0c2d1e677dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.079116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e21b17-ab5b-4288-a338-0c2d1e677dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.081200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e21b17-ab5b-4288-a338-0c2d1e677dd8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.096855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 885/1790] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:39.100345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837db104-51aa-4696-8503-95e1d513b3ba admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.115603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837db104-51aa-4696-8503-95e1d513b3ba admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.117148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b23a40-ee64-440e-8c5d-6c0c91d6fbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.117873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23a40-ee64-440e-8c5d-6c0c91d6fbe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.118154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23a40-ee64-440e-8c5d-6c0c91d6fbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.118438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23a40-ee64-440e-8c5d-6c0c91d6fbe3 admin admin] PATH_INFO: `/v3/projects/775b89b6f18948d58cb96d4593951158/users/a43147c973d2413d95aa30792ab0ef16/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.138179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b23a40-ee64-440e-8c5d-6c0c91d6fbe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=775b89b6f18948d58cb96d4593951158, user_id=a43147c973d2413d95aa30792ab0ef16, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.140856 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b23a40-ee64-440e-8c5d-6c0c91d6fbe3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.244754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 906/1791] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:39 2022] PUT /identity/v3/projects/775b89b6f18948d58cb96d4593951158/users/a43147c973d2413d95aa30792ab0ef16/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:39.256984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5253098-3dde-4eb3-a4b4-96192cd60ae2 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.257276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5253098-3dde-4eb3-a4b4-96192cd60ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.257559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5253098-3dde-4eb3-a4b4-96192cd60ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.291626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b5253098-3dde-4eb3-a4b4-96192cd60ae2 None None] MFA Rules not processed for user `a43147c973d2413d95aa30792ab0ef16`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:39.292926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b5253098-3dde-4eb3-a4b4-96192cd60ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:39.328221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:39.378442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 886/1792] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:39.383258 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b23a40-ee64-440e-8c5d-6c0c91d6fbe3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.478600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b23a40-ee64-440e-8c5d-6c0c91d6fbe3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.483643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb7d48-887c-4f55-9078-02ee45453050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.484327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb7d48-887c-4f55-9078-02ee45453050 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.484492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb7d48-887c-4f55-9078-02ee45453050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.484790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb7d48-887c-4f55-9078-02ee45453050 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.485436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb7d48-887c-4f55-9078-02ee45453050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.487926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb7d48-887c-4f55-9078-02ee45453050 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.616151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 907/1793] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:39 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:39.616729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5253098-3dde-4eb3-a4b4-96192cd60ae2 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.629418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5253098-3dde-4eb3-a4b4-96192cd60ae2 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.688021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f68749-cbcd-4e97-a800-718757ecdeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.688640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f68749-cbcd-4e97-a800-718757ecdeae admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.688920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f68749-cbcd-4e97-a800-718757ecdeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.689191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f68749-cbcd-4e97-a800-718757ecdeae admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.690419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f68749-cbcd-4e97-a800-718757ecdeae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.692334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f68749-cbcd-4e97-a800-718757ecdeae admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.763829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 887/1794] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:39 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:39.767878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb7d48-887c-4f55-9078-02ee45453050 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.778555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb7d48-887c-4f55-9078-02ee45453050 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.783403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed691-9014-4f20-9305-20ec965a5d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.784003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed691-9014-4f20-9305-20ec965a5d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.784283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed691-9014-4f20-9305-20ec965a5d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.784557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed691-9014-4f20-9305-20ec965a5d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.785207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ed691-9014-4f20-9305-20ec965a5d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.786830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ed691-9014-4f20-9305-20ec965a5d99 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.797837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 908/1795] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:39.802786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f68749-cbcd-4e97-a800-718757ecdeae admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.812955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f68749-cbcd-4e97-a800-718757ecdeae admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.818292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9a311-a6f7-435f-a53e-2f436f64a8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.818887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9a311-a6f7-435f-a53e-2f436f64a8af admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.819172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9a311-a6f7-435f-a53e-2f436f64a8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.819471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9a311-a6f7-435f-a53e-2f436f64a8af admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.820123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9a311-a6f7-435f-a53e-2f436f64a8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.821781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9a311-a6f7-435f-a53e-2f436f64a8af admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.831950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 888/1796] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:39.836644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed691-9014-4f20-9305-20ec965a5d99 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.841998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:39.847602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed691-9014-4f20-9305-20ec965a5d99 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.855967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96adc0-c3b9-42cb-a192-ed907e628af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.855967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96adc0-c3b9-42cb-a192-ed907e628af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.855967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96adc0-c3b9-42cb-a192-ed907e628af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.855967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96adc0-c3b9-42cb-a192-ed907e628af6 admin admin] PATH_INFO: `/v3/projects/b113a0e20804437a8f18a67b0d2a9b13/users/b4584607b4544893984c0aa3e20fef43/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.869885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96adc0-c3b9-42cb-a192-ed907e628af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b113a0e20804437a8f18a67b0d2a9b13, user_id=b4584607b4544893984c0aa3e20fef43, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.872522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96adc0-c3b9-42cb-a192-ed907e628af6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.953812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 909/1797] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:39 2022] PUT /identity/v3/projects/b113a0e20804437a8f18a67b0d2a9b13/users/b4584607b4544893984c0aa3e20fef43/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:39.963988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414fdbd-87af-42a3-8767-76ce0de3e548 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.964377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414fdbd-87af-42a3-8767-76ce0de3e548 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.967367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414fdbd-87af-42a3-8767-76ce0de3e548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.994524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-5414fdbd-87af-42a3-8767-76ce0de3e548 None None] MFA Rules not processed for user `b4584607b4544893984c0aa3e20fef43`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:39.995721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5414fdbd-87af-42a3-8767-76ce0de3e548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:40.063062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 889/1798] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:40.068319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96adc0-c3b9-42cb-a192-ed907e628af6 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:40.134348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96adc0-c3b9-42cb-a192-ed907e628af6 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:40.139480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093bc1f-d361-4d9c-b8e0-4a2694045866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:40.139978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093bc1f-d361-4d9c-b8e0-4a2694045866 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:40.140261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093bc1f-d361-4d9c-b8e0-4a2694045866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:40.140536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093bc1f-d361-4d9c-b8e0-4a2694045866 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:40.141170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093bc1f-d361-4d9c-b8e0-4a2694045866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:40.143897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093bc1f-d361-4d9c-b8e0-4a2694045866 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:40.250137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 910/1799] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:40 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:40.251397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414fdbd-87af-42a3-8767-76ce0de3e548 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:40.264852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414fdbd-87af-42a3-8767-76ce0de3e548 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:40.302312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:40.332958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f94056-226f-421a-b96c-6fdfd2147c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:40.333801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94056-226f-421a-b96c-6fdfd2147c90 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:40.334146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94056-226f-421a-b96c-6fdfd2147c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:40.334444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94056-226f-421a-b96c-6fdfd2147c90 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:40.335095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f94056-226f-421a-b96c-6fdfd2147c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:40.337193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f94056-226f-421a-b96c-6fdfd2147c90 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:40.436554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 890/1800] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:40 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:40.448983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093bc1f-d361-4d9c-b8e0-4a2694045866 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:40.461951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093bc1f-d361-4d9c-b8e0-4a2694045866 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:40.467044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595a380-10bb-4374-ac23-9437caad3e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:40.467680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a380-10bb-4374-ac23-9437caad3e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:40.467968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a380-10bb-4374-ac23-9437caad3e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:40.468240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a380-10bb-4374-ac23-9437caad3e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:40.468913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a380-10bb-4374-ac23-9437caad3e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:40.470836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a380-10bb-4374-ac23-9437caad3e18 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:40.485214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 911/1801] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:40.491418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f94056-226f-421a-b96c-6fdfd2147c90 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:40.507189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f94056-226f-421a-b96c-6fdfd2147c90 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:40.514015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb1e8-873b-4fae-aaef-409569f85189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:40.514812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb1e8-873b-4fae-aaef-409569f85189 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:40.515291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb1e8-873b-4fae-aaef-409569f85189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:40.519711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb1e8-873b-4fae-aaef-409569f85189 admin admin] PATH_INFO: `/v3/projects/ab0909c833f144fcb5b6ab5faa515ff5/users/5542183c6cbc40259c1c105bf02c5c82/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:40.570992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb1e8-873b-4fae-aaef-409569f85189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0909c833f144fcb5b6ab5faa515ff5, user_id=5542183c6cbc40259c1c105bf02c5c82, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:40.573745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb1e8-873b-4fae-aaef-409569f85189 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:40.703457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 891/1802] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:40 2022] PUT /identity/v3/projects/ab0909c833f144fcb5b6ab5faa515ff5/users/5542183c6cbc40259c1c105bf02c5c82/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:40.723875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595a380-10bb-4374-ac23-9437caad3e18 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:40.737878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595a380-10bb-4374-ac23-9437caad3e18 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:40.822668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d70c1c-dd0b-4c63-8ce0-81ee7d48c897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:40.823680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d70c1c-dd0b-4c63-8ce0-81ee7d48c897 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:40.824296 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d70c1c-dd0b-4c63-8ce0-81ee7d48c897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:40.824831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d70c1c-dd0b-4c63-8ce0-81ee7d48c897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:40.825657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d70c1c-dd0b-4c63-8ce0-81ee7d48c897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:40.828095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d70c1c-dd0b-4c63-8ce0-81ee7d48c897 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:40.851973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 912/1803] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:40.856129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb1e8-873b-4fae-aaef-409569f85189 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:40.884180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb1e8-873b-4fae-aaef-409569f85189 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:40.898203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:40.898899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:40.899273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:40.899559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] PATH_INFO: `/v3/projects/ab0909c833f144fcb5b6ab5faa515ff5/users/5542183c6cbc40259c1c105bf02c5c82/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:40.901686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0909c833f144fcb5b6ab5faa515ff5, user_id=5542183c6cbc40259c1c105bf02c5c82, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:40.904461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:40.918629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 892/1804] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:40 2022] PUT /identity/v3/projects/ab0909c833f144fcb5b6ab5faa515ff5/users/5542183c6cbc40259c1c105bf02c5c82/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:40.938189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d25f-14ba-4591-ad20-89af32ef66bb None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:40.938473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d25f-14ba-4591-ad20-89af32ef66bb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:40.938781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d25f-14ba-4591-ad20-89af32ef66bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:40.997794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-fcd5d25f-14ba-4591-ad20-89af32ef66bb None None] MFA Rules not processed for user `5542183c6cbc40259c1c105bf02c5c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:40.999078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-fcd5d25f-14ba-4591-ad20-89af32ef66bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:41.081240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:41.129040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 913/1805] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:41.141819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:41.144281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:41.266409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154418a3-bdaf-48e4-a8e5-8aea9d193348 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:41.272579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:41.274182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:41.274808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:41.275608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:41.291446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:41.292961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:41.320591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 893/1806] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:41.391090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5d25f-14ba-4591-ad20-89af32ef66bb None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:41.406537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5d25f-14ba-4591-ad20-89af32ef66bb None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:41.420713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:41.421387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:41.421722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:41.422113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:41.424677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:41.525989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:41.527735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:41.544637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 914/1807] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:43.401193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:43.402786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:43.504341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453744e9-3376-42a5-a170-0f0473f7b383 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:43.510425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:43.511125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:43.511415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:43.511996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] PATH_INFO: `/v3/users/ad2f353369fa4c50a108072c6bb986d7` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:43.512970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2f353369fa4c50a108072c6bb986d7)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:43.515079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:43.626583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:43.628706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:43.641260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4f055-8192-4dd6-a1b9-a2524ce47507 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:43.649735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd326ed-778d-40a4-b103-cdc8dce6af56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:43.649735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd326ed-778d-40a4-b103-cdc8dce6af56 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:43.649735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd326ed-778d-40a4-b103-cdc8dce6af56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:43.649735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd326ed-778d-40a4-b103-cdc8dce6af56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:43.711068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd326ed-778d-40a4-b103-cdc8dce6af56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:43.712552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd326ed-778d-40a4-b103-cdc8dce6af56 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:43.722902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 915/1808] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:43.737513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:43.750626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2f353369fa4c50a108072c6bb986d7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:43.832373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2f353369fa4c50a108072c6bb986d7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:43.833306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2f353369fa4c50a108072c6bb986d7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:43.842057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2f353369fa4c50a108072c6bb986d7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:43.842847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2f353369fa4c50a108072c6bb986d7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:43.863336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 894/1809] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:43 2022] DELETE /identity/v3/users/ad2f353369fa4c50a108072c6bb986d7 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:43.868085 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef255b5-f8e7-477a-9fb7-bece199c7f8b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:43.868371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef255b5-f8e7-477a-9fb7-bece199c7f8b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:43.868647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef255b5-f8e7-477a-9fb7-bece199c7f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:43.889285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1ef255b5-f8e7-477a-9fb7-bece199c7f8b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:43.890324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1ef255b5-f8e7-477a-9fb7-bece199c7f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:43.963815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 916/1810] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:43.975443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:43.980939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:44.042782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb2503-8655-4bae-bc22-4909260e71d5 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:44.046412 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19300e8-dba6-4b81-859b-89fe7c62a379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:44.047018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19300e8-dba6-4b81-859b-89fe7c62a379 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:44.047299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19300e8-dba6-4b81-859b-89fe7c62a379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:44.047573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19300e8-dba6-4b81-859b-89fe7c62a379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:44.058735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19300e8-dba6-4b81-859b-89fe7c62a379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:44.060359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19300e8-dba6-4b81-859b-89fe7c62a379 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:44.070915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 895/1811] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:44.573980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef255b5-f8e7-477a-9fb7-bece199c7f8b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:44.578596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1ef255b5-f8e7-477a-9fb7-bece199c7f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:44.596156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef255b5-f8e7-477a-9fb7-bece199c7f8b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:44.601794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:44.604557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:44.604869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:44.605150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] PATH_INFO: `/v3/projects/eae14007e8564c9686c0d28f8698f342` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:44.606833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae14007e8564c9686c0d28f8698f342)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:44.608972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:44.610738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:44.749621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae14007e8564c9686c0d28f8698f342', 'request_id': 'req-f700f8ed-6317-458f-9e3c-93ea41270bd6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:44.821233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae14007e8564c9686c0d28f8698f342', 'request_id': 'req-f700f8ed-6317-458f-9e3c-93ea41270bd6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:44.830482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae14007e8564c9686c0d28f8698f342', 'request_id': 'req-f700f8ed-6317-458f-9e3c-93ea41270bd6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:44.841611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] The token cache is being invalidate because project eae14007e8564c9686c0d28f8698f342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:44.841978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:44.842588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:44.844338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 917/1812] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:44 2022] DELETE /identity/v3/projects/eae14007e8564c9686c0d28f8698f342 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:44.849328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19300e8-dba6-4b81-859b-89fe7c62a379 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:44.866185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19300e8-dba6-4b81-859b-89fe7c62a379 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:44.928815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:44.929482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:44.929770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:44.930085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] PATH_INFO: `/v3/users/5b28a18a4aee4e298dd6dabb34938bd9` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:44.931238 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b28a18a4aee4e298dd6dabb34938bd9)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:44.933168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:45.109489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b28a18a4aee4e298dd6dabb34938bd9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:45.169458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b28a18a4aee4e298dd6dabb34938bd9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:45.170214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b28a18a4aee4e298dd6dabb34938bd9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:45.171900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:45.181254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b28a18a4aee4e298dd6dabb34938bd9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:45.182365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-49839f17-3067-4ac7-aeb0-a7fa35d64f87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b28a18a4aee4e298dd6dabb34938bd9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:45.202426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 896/1813] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:44 2022] DELETE /identity/v3/users/5b28a18a4aee4e298dd6dabb34938bd9 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:45.709278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:45.711151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:45.738037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700f8ed-6317-458f-9e3c-93ea41270bd6 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:45.819554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:45.820288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:45.820527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:45.820919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] PATH_INFO: `/v3/projects/0b19b0550c95483593f5e79669e881cc` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:45.822191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b19b0550c95483593f5e79669e881cc)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:45.824309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:45.951168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b19b0550c95483593f5e79669e881cc', 'request_id': 'req-9f5d815e-3cd5-4b45-95f2-87372e255bcf'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:46.055396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b19b0550c95483593f5e79669e881cc', 'request_id': 'req-9f5d815e-3cd5-4b45-95f2-87372e255bcf'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:46.074442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b19b0550c95483593f5e79669e881cc', 'request_id': 'req-9f5d815e-3cd5-4b45-95f2-87372e255bcf'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:46.088852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] The token cache is being invalidate because project 0b19b0550c95483593f5e79669e881cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:46.090494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:46.090494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9f5d815e-3cd5-4b45-95f2-87372e255bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:46.091515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 918/1814] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:45 2022] DELETE /identity/v3/projects/0b19b0550c95483593f5e79669e881cc => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:46.096971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d603-3724-494d-8452-20ffcf2243cc None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.097475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d603-3724-494d-8452-20ffcf2243cc None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.097553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d603-3724-494d-8452-20ffcf2243cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.118146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f748d603-3724-494d-8452-20ffcf2243cc None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:46.119493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f748d603-3724-494d-8452-20ffcf2243cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:46.194666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 897/1815] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:46.203129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ebe76-958c-4327-b903-878b3e45731c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.203559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ebe76-958c-4327-b903-878b3e45731c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.203956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ebe76-958c-4327-b903-878b3e45731c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.228977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-626ebe76-958c-4327-b903-878b3e45731c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:46.230286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-626ebe76-958c-4327-b903-878b3e45731c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:46.239412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 919/1816] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:46.243966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748d603-3724-494d-8452-20ffcf2243cc None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:46.256250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748d603-3724-494d-8452-20ffcf2243cc None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:46.261234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:46.261871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.262159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.262438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.263263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:46.263579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:46.265225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:46.279850 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 898/1817] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:46.286279 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ebe76-958c-4327-b903-878b3e45731c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:46.301681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ebe76-958c-4327-b903-878b3e45731c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:46.309646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50882b1-8adc-479e-b590-726725055ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:46.310830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50882b1-8adc-479e-b590-726725055ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.311411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50882b1-8adc-479e-b590-726725055ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.311919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50882b1-8adc-479e-b590-726725055ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.313122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50882b1-8adc-479e-b590-726725055ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:46.316962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50882b1-8adc-479e-b590-726725055ddd admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:46.337257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:46.437499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 920/1818] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:46 2022] POST /identity/v3/projects => generated 391 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:46.438989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:46.449424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b49d1-66b6-437d-be23-0a6ea4d7bc30 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:46.546701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7e876-d89f-401f-a57a-8a26210221af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:46.546827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7e876-d89f-401f-a57a-8a26210221af admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.547117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7e876-d89f-401f-a57a-8a26210221af admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.547396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7e876-d89f-401f-a57a-8a26210221af admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.548780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7e876-d89f-401f-a57a-8a26210221af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:46.551021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7e876-d89f-401f-a57a-8a26210221af admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:46.728975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 899/1819] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:46 2022] POST /identity/v3/users => generated 296 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:46.733286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50882b1-8adc-479e-b590-726725055ddd admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:46.746072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50882b1-8adc-479e-b590-726725055ddd admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:46.751561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1fe1-df44-47c9-87c8-814bec424c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:46.753425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1fe1-df44-47c9-87c8-814bec424c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.754389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1fe1-df44-47c9-87c8-814bec424c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.754678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1fe1-df44-47c9-87c8-814bec424c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.755371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac1fe1-df44-47c9-87c8-814bec424c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:46.757026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac1fe1-df44-47c9-87c8-814bec424c32 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:46.769994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 921/1820] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:46.774183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7e876-d89f-401f-a57a-8a26210221af admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:46.789409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7e876-d89f-401f-a57a-8a26210221af admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:46.796434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9567e-8cf2-44bf-8c54-325a0f0f57db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:46.796630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9567e-8cf2-44bf-8c54-325a0f0f57db admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.796897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9567e-8cf2-44bf-8c54-325a0f0f57db admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.797920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9567e-8cf2-44bf-8c54-325a0f0f57db admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.799487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9567e-8cf2-44bf-8c54-325a0f0f57db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:46.801364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9567e-8cf2-44bf-8c54-325a0f0f57db admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:46.818711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 900/1821] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:46.821448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1fe1-df44-47c9-87c8-814bec424c32 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:46.839793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1fe1-df44-47c9-87c8-814bec424c32 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:46.845708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37fbec-917f-4d4b-8d6d-8a6907c3d7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:46.848087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37fbec-917f-4d4b-8d6d-8a6907c3d7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.848087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37fbec-917f-4d4b-8d6d-8a6907c3d7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.848087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37fbec-917f-4d4b-8d6d-8a6907c3d7f6 admin admin] PATH_INFO: `/v3/projects/07479fe65ea94b02a9678fa514e559d2/users/03b7e35f90564ae89855fd98c125315c/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.868668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37fbec-917f-4d4b-8d6d-8a6907c3d7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07479fe65ea94b02a9678fa514e559d2, user_id=03b7e35f90564ae89855fd98c125315c, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:46.871809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37fbec-917f-4d4b-8d6d-8a6907c3d7f6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:46.953562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 922/1822] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:46 2022] PUT /identity/v3/projects/07479fe65ea94b02a9678fa514e559d2/users/03b7e35f90564ae89855fd98c125315c/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:46.966287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff59c6-af5b-490b-9701-6aecf9a2840e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:46.966497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff59c6-af5b-490b-9701-6aecf9a2840e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:46.966775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff59c6-af5b-490b-9701-6aecf9a2840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:46.999993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a2ff59c6-af5b-490b-9701-6aecf9a2840e None None] MFA Rules not processed for user `03b7e35f90564ae89855fd98c125315c`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:47.001252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a2ff59c6-af5b-490b-9701-6aecf9a2840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:47.032086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:47.110047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 901/1823] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:47.111735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37fbec-917f-4d4b-8d6d-8a6907c3d7f6 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:47.217196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37fbec-917f-4d4b-8d6d-8a6907c3d7f6 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:47.223377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a862c-2d9f-4d97-aaaa-4b5b67f2305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:47.224093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a862c-2d9f-4d97-aaaa-4b5b67f2305b admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.224543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a862c-2d9f-4d97-aaaa-4b5b67f2305b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.225087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a862c-2d9f-4d97-aaaa-4b5b67f2305b admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.225808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a862c-2d9f-4d97-aaaa-4b5b67f2305b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:47.227892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a862c-2d9f-4d97-aaaa-4b5b67f2305b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:47.337386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 923/1824] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:47 2022] POST /identity/v3/projects => generated 393 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:47.341601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff59c6-af5b-490b-9701-6aecf9a2840e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:47.353557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff59c6-af5b-490b-9701-6aecf9a2840e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:47.413381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ac9d-1674-41ca-a4dd-f6ee28b489e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:47.414055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ac9d-1674-41ca-a4dd-f6ee28b489e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.417014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ac9d-1674-41ca-a4dd-f6ee28b489e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.417014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ac9d-1674-41ca-a4dd-f6ee28b489e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.417014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8ac9d-1674-41ca-a4dd-f6ee28b489e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:47.417254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8ac9d-1674-41ca-a4dd-f6ee28b489e5 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:47.483693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 902/1825] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:47 2022] POST /identity/v3/users => generated 303 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:47.487852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a862c-2d9f-4d97-aaaa-4b5b67f2305b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:47.498593 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a862c-2d9f-4d97-aaaa-4b5b67f2305b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:47.504377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee18968-9d95-4b16-b6c0-b309d30b4ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:47.504989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee18968-9d95-4b16-b6c0-b309d30b4ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.505291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee18968-9d95-4b16-b6c0-b309d30b4ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.509040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee18968-9d95-4b16-b6c0-b309d30b4ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.509040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee18968-9d95-4b16-b6c0-b309d30b4ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:47.509040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee18968-9d95-4b16-b6c0-b309d30b4ea8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:47.524762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 924/1826] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:47.526098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ac9d-1674-41ca-a4dd-f6ee28b489e5 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:47.539884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ac9d-1674-41ca-a4dd-f6ee28b489e5 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:47.546209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe0bb5-c917-4394-ab87-8b13c0aa8ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:47.546889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0bb5-c917-4394-ab87-8b13c0aa8ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.547171 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0bb5-c917-4394-ab87-8b13c0aa8ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.547449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0bb5-c917-4394-ab87-8b13c0aa8ab3 admin admin] PATH_INFO: `/v3/projects/76c075d771084a90a73e8f27042c1f3d/users/fe3e49413fd2430dbbd49465e7faec61/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.571731 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe0bb5-c917-4394-ab87-8b13c0aa8ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c075d771084a90a73e8f27042c1f3d, user_id=fe3e49413fd2430dbbd49465e7faec61, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:47.575224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe0bb5-c917-4394-ab87-8b13c0aa8ab3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:47.736795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 903/1827] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:47 2022] PUT /identity/v3/projects/76c075d771084a90a73e8f27042c1f3d/users/fe3e49413fd2430dbbd49465e7faec61/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:47.741193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee18968-9d95-4b16-b6c0-b309d30b4ea8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:47.753963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee18968-9d95-4b16-b6c0-b309d30b4ea8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:47.824234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf84673-f295-4eb1-a120-d3bc34b9cd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:47.824995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf84673-f295-4eb1-a120-d3bc34b9cd88 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.825308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf84673-f295-4eb1-a120-d3bc34b9cd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.825623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf84673-f295-4eb1-a120-d3bc34b9cd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.826377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf84673-f295-4eb1-a120-d3bc34b9cd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:47.829148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf84673-f295-4eb1-a120-d3bc34b9cd88 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:47.841341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 925/1828] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:47.846791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe0bb5-c917-4394-ab87-8b13c0aa8ab3 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:47.863471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe0bb5-c917-4394-ab87-8b13c0aa8ab3 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:47.869837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:47.870857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.871143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.871604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] PATH_INFO: `/v3/projects/76c075d771084a90a73e8f27042c1f3d/users/fe3e49413fd2430dbbd49465e7faec61/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.873965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c075d771084a90a73e8f27042c1f3d, user_id=fe3e49413fd2430dbbd49465e7faec61, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:47.879470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:47.894407 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 904/1829] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:47 2022] PUT /identity/v3/projects/76c075d771084a90a73e8f27042c1f3d/users/fe3e49413fd2430dbbd49465e7faec61/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:47.905594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81186ecd-5a0f-4982-9568-c42650391612 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.906064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81186ecd-5a0f-4982-9568-c42650391612 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.906343 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81186ecd-5a0f-4982-9568-c42650391612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.938077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-81186ecd-5a0f-4982-9568-c42650391612 None None] MFA Rules not processed for user `fe3e49413fd2430dbbd49465e7faec61`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:47.940088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-81186ecd-5a0f-4982-9568-c42650391612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:47.954570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:48.016658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 926/1830] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:47 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:48.027860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.029603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:48.098932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbf549-3f8e-4367-8323-e290e60a351e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.102640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81552a47-8f0a-471e-9ead-39d70632973e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.103259 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81552a47-8f0a-471e-9ead-39d70632973e service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.103559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81552a47-8f0a-471e-9ead-39d70632973e service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.103830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81552a47-8f0a-471e-9ead-39d70632973e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.114020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81552a47-8f0a-471e-9ead-39d70632973e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.115465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81552a47-8f0a-471e-9ead-39d70632973e service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.126056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 905/1831] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:48 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:48.178800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81186ecd-5a0f-4982-9568-c42650391612 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.189893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81186ecd-5a0f-4982-9568-c42650391612 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.194082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2517-30d1-450a-a782-c1e1a286a4c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.194859 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2517-30d1-450a-a782-c1e1a286a4c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.195255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2517-30d1-450a-a782-c1e1a286a4c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.195626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2517-30d1-450a-a782-c1e1a286a4c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.271259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2517-30d1-450a-a782-c1e1a286a4c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.272734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2517-30d1-450a-a782-c1e1a286a4c7 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.284136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 927/1832] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:52.399385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5640a9-357c-490d-8e4e-cfd86ec03d8f None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.400492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5640a9-357c-490d-8e4e-cfd86ec03d8f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.401055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5640a9-357c-490d-8e4e-cfd86ec03d8f None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.403592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 906/1833] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:49:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:52.436264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc29a-ac30-4c64-860f-8c031ab6478a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.438149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc29a-ac30-4c64-860f-8c031ab6478a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.438263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc29a-ac30-4c64-860f-8c031ab6478a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.487382 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-21fbc29a-ac30-4c64-860f-8c031ab6478a None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:52.490883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-21fbc29a-ac30-4c64-860f-8c031ab6478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:52.691585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 928/1834] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:49:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:52.716484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5640a9-357c-490d-8e4e-cfd86ec03d8f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.718271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5c5640a9-357c-490d-8e4e-cfd86ec03d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:52.766175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5640a9-357c-490d-8e4e-cfd86ec03d8f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.794385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a04b1-6cbe-4dd7-a8f7-7253b8a9583f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.795142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a04b1-6cbe-4dd7-a8f7-7253b8a9583f service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.795562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a04b1-6cbe-4dd7-a8f7-7253b8a9583f service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.795960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a04b1-6cbe-4dd7-a8f7-7253b8a9583f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.811816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a04b1-6cbe-4dd7-a8f7-7253b8a9583f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.814971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a04b1-6cbe-4dd7-a8f7-7253b8a9583f service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.842000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 907/1835] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:57.467942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbc29a-ac30-4c64-860f-8c031ab6478a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:57.469399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-21fbc29a-ac30-4c64-860f-8c031ab6478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:57.486846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbc29a-ac30-4c64-860f-8c031ab6478a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:57.490685 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b114c-08c6-4848-a2cb-d21148ae0fcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:57.491303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b114c-08c6-4848-a2cb-d21148ae0fcc service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:57.491586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b114c-08c6-4848-a2cb-d21148ae0fcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:57.491862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b114c-08c6-4848-a2cb-d21148ae0fcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:57.493722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5f6b114c-08c6-4848-a2cb-d21148ae0fcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:57.566226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:57.587917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b114c-08c6-4848-a2cb-d21148ae0fcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:57.589385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b114c-08c6-4848-a2cb-d21148ae0fcc service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:57.599900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 929/1836] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:57.684660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0380a-a8e0-407f-8281-3c9aea644413 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:57.685265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0380a-a8e0-407f-8281-3c9aea644413 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:57.685265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0380a-a8e0-407f-8281-3c9aea644413 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:57.689273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 908/1837] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:49:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:57.700180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1410-9e1a-4600-ba72-5826be5b0be4 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:57.700711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1410-9e1a-4600-ba72-5826be5b0be4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:57.701063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1410-9e1a-4600-ba72-5826be5b0be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:57.722555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-aceb1410-9e1a-4600-ba72-5826be5b0be4 None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:57.723780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-aceb1410-9e1a-4600-ba72-5826be5b0be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:57.733348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 930/1838] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:49:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:57.747259 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0380a-a8e0-407f-8281-3c9aea644413 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:57.749079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-79b0380a-a8e0-407f-8281-3c9aea644413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:57.771287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0380a-a8e0-407f-8281-3c9aea644413 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:57.778846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde2fb3-7161-4e86-8d79-28d15fce6b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:57.779660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2fb3-7161-4e86-8d79-28d15fce6b0a service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:57.780268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2fb3-7161-4e86-8d79-28d15fce6b0a service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:57.780546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2fb3-7161-4e86-8d79-28d15fce6b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:57.790289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:57.793193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde2fb3-7161-4e86-8d79-28d15fce6b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:57.795431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde2fb3-7161-4e86-8d79-28d15fce6b0a service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:57.808160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 909/1839] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:57.963841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1410-9e1a-4600-ba72-5826be5b0be4 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:58.064528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1410-9e1a-4600-ba72-5826be5b0be4 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:58.070320 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:58.071088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.071466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.071668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] PATH_INFO: `/v3/users/03b7e35f90564ae89855fd98c125315c` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.072811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b7e35f90564ae89855fd98c125315c)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:58.074768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:58.280921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b7e35f90564ae89855fd98c125315c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.296254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b7e35f90564ae89855fd98c125315c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.421336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b7e35f90564ae89855fd98c125315c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.422451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b7e35f90564ae89855fd98c125315c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.424307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b7e35f90564ae89855fd98c125315c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.444786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 931/1840] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:57 2022] DELETE /identity/v3/users/03b7e35f90564ae89855fd98c125315c => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:58.451463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5f22-6fad-4825-bc71-7663bc19f6ca None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.453422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5f22-6fad-4825-bc71-7663bc19f6ca None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.454207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5f22-6fad-4825-bc71-7663bc19f6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.499488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-16cd5f22-6fad-4825-bc71-7663bc19f6ca None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:58.503735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-16cd5f22-6fad-4825-bc71-7663bc19f6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:58.646303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 910/1841] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:58.646385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:58.647834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:58.756875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727f973-ad71-42bf-9c03-6f13173a6d85 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:58.762084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce3fba-a6c0-4f1a-bffb-34a40489a0ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:58.766247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce3fba-a6c0-4f1a-bffb-34a40489a0ae service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.767958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce3fba-a6c0-4f1a-bffb-34a40489a0ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.768329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce3fba-a6c0-4f1a-bffb-34a40489a0ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.779290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:58.786523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce3fba-a6c0-4f1a-bffb-34a40489a0ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:58.787574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce3fba-a6c0-4f1a-bffb-34a40489a0ae service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:58.804421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 932/1842] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:59.527279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5f22-6fad-4825-bc71-7663bc19f6ca None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:59.529252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-16cd5f22-6fad-4825-bc71-7663bc19f6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:59.555165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5f22-6fad-4825-bc71-7663bc19f6ca None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:59.570749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:59.573100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:59.573100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:59.573100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] PATH_INFO: `/v3/projects/07479fe65ea94b02a9678fa514e559d2` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:59.573722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=07479fe65ea94b02a9678fa514e559d2)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:59.575275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:59.802738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07479fe65ea94b02a9678fa514e559d2', 'request_id': 'req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:59.925328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07479fe65ea94b02a9678fa514e559d2', 'request_id': 'req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:59.933390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07479fe65ea94b02a9678fa514e559d2', 'request_id': 'req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:59.951386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] The token cache is being invalidate because project 07479fe65ea94b02a9678fa514e559d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:59.952179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:59.953556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:59.958695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 911/1843] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:59 2022] DELETE /identity/v3/projects/07479fe65ea94b02a9678fa514e559d2 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:59.966384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce3fba-a6c0-4f1a-bffb-34a40489a0ae service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:59.977576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce3fba-a6c0-4f1a-bffb-34a40489a0ae service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:00.040481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:00.043509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:00.043509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:00.043509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] PATH_INFO: `/v3/users/fe3e49413fd2430dbbd49465e7faec61` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:00.043509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3e49413fd2430dbbd49465e7faec61)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:00.044715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:00.269886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3e49413fd2430dbbd49465e7faec61'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:00.284527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3e49413fd2430dbbd49465e7faec61'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:00.364367 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3e49413fd2430dbbd49465e7faec61'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:00.365007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3e49413fd2430dbbd49465e7faec61'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:00.365631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-56e42867-c0c9-45eb-b89f-0bdf61086b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3e49413fd2430dbbd49465e7faec61'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:00.380591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 933/1844] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:59 2022] DELETE /identity/v3/users/fe3e49413fd2430dbbd49465e7faec61 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:00.879863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:00.881348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:00.895294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6653c22-2f8f-43bb-9a42-d3e12caf27ed admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:00.982213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:00.982980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:00.983480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:00.983790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] PATH_INFO: `/v3/projects/76c075d771084a90a73e8f27042c1f3d` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:00.986340 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c075d771084a90a73e8f27042c1f3d)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:00.989267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:01.029612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:01.188849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c075d771084a90a73e8f27042c1f3d', 'request_id': 'req-e30ea434-0879-4336-9587-e4b6876a73c2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.271978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c075d771084a90a73e8f27042c1f3d', 'request_id': 'req-e30ea434-0879-4336-9587-e4b6876a73c2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.278443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c075d771084a90a73e8f27042c1f3d', 'request_id': 'req-e30ea434-0879-4336-9587-e4b6876a73c2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.292294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] The token cache is being invalidate because project 76c075d771084a90a73e8f27042c1f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:01.292586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.293330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e30ea434-0879-4336-9587-e4b6876a73c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.295188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 912/1845] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:00 2022] DELETE /identity/v3/projects/76c075d771084a90a73e8f27042c1f3d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:01.300668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bdbe-edf9-4b1e-acb7-055131f3000e None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.304668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bdbe-edf9-4b1e-acb7-055131f3000e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.304668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bdbe-edf9-4b1e-acb7-055131f3000e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.326517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ce54bdbe-edf9-4b1e-acb7-055131f3000e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:01.328064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ce54bdbe-edf9-4b1e-acb7-055131f3000e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:01.401414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 934/1846] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:01.410253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e03f-71bb-452a-8fe8-91a3e65cb988 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.410553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e03f-71bb-452a-8fe8-91a3e65cb988 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.410815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e03f-71bb-452a-8fe8-91a3e65cb988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.433099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-aba7e03f-71bb-452a-8fe8-91a3e65cb988 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:01.435424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-aba7e03f-71bb-452a-8fe8-91a3e65cb988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:01.444722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 913/1847] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:01.451113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54bdbe-edf9-4b1e-acb7-055131f3000e None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:01.465305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54bdbe-edf9-4b1e-acb7-055131f3000e None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:01.473837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:01.474887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.475367 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.476234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.477035 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:01.477563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:01.481087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:01.503129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 935/1848] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:01.507559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7e03f-71bb-452a-8fe8-91a3e65cb988 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:01.517793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7e03f-71bb-452a-8fe8-91a3e65cb988 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:01.522676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cf0c3-752a-4389-821a-e3731cd276f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:01.533250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cf0c3-752a-4389-821a-e3731cd276f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.533250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cf0c3-752a-4389-821a-e3731cd276f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.533250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cf0c3-752a-4389-821a-e3731cd276f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.533250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cf0c3-752a-4389-821a-e3731cd276f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:01.533250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cf0c3-752a-4389-821a-e3731cd276f8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:01.707722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 914/1849] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:01 2022] POST /identity/v3/projects => generated 389 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:01.708521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:01.713796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:01.719932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce26f50-ffb3-45fa-b305-719396fd31c6 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:01.785319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a00541-d825-45b8-b474-9b1ceb8b9c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:01.785998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a00541-d825-45b8-b474-9b1ceb8b9c7a admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.786282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a00541-d825-45b8-b474-9b1ceb8b9c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.786558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a00541-d825-45b8-b474-9b1ceb8b9c7a admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.787202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a00541-d825-45b8-b474-9b1ceb8b9c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:01.789124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a00541-d825-45b8-b474-9b1ceb8b9c7a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:01.929874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 936/1850] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:01 2022] POST /identity/v3/users => generated 295 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:01.934218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cf0c3-752a-4389-821a-e3731cd276f8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:01.945356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cf0c3-752a-4389-821a-e3731cd276f8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:01.950357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a35a1-a4d7-478b-aafb-e8cd9f9598f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:01.950974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a35a1-a4d7-478b-aafb-e8cd9f9598f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.951255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a35a1-a4d7-478b-aafb-e8cd9f9598f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.951687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a35a1-a4d7-478b-aafb-e8cd9f9598f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.952351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a35a1-a4d7-478b-aafb-e8cd9f9598f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:01.954064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a35a1-a4d7-478b-aafb-e8cd9f9598f6 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:01.965776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 915/1851] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:01.970663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a00541-d825-45b8-b474-9b1ceb8b9c7a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:01.982062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a00541-d825-45b8-b474-9b1ceb8b9c7a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:01.988170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b677fb-5d15-4272-a1b7-143d5ac0b4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:01.988985 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b677fb-5d15-4272-a1b7-143d5ac0b4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.989356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b677fb-5d15-4272-a1b7-143d5ac0b4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.989728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b677fb-5d15-4272-a1b7-143d5ac0b4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.990621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b677fb-5d15-4272-a1b7-143d5ac0b4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:01.992405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b677fb-5d15-4272-a1b7-143d5ac0b4f8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.004206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 937/1852] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:02.008536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a35a1-a4d7-478b-aafb-e8cd9f9598f6 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.019756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a35a1-a4d7-478b-aafb-e8cd9f9598f6 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.024706 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228cae8-c7a3-443c-a32d-c72e69359fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.025401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cae8-c7a3-443c-a32d-c72e69359fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.025683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cae8-c7a3-443c-a32d-c72e69359fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.026020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cae8-c7a3-443c-a32d-c72e69359fc3 admin admin] PATH_INFO: `/v3/projects/87c6230df86c4e1b96f1bbe34809443f/users/a4c425fce3e94daf89d14a220b761a7f/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.045673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228cae8-c7a3-443c-a32d-c72e69359fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c6230df86c4e1b96f1bbe34809443f, user_id=a4c425fce3e94daf89d14a220b761a7f, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.048965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228cae8-c7a3-443c-a32d-c72e69359fc3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.182369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 916/1853] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:02 2022] PUT /identity/v3/projects/87c6230df86c4e1b96f1bbe34809443f/users/a4c425fce3e94daf89d14a220b761a7f/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:02.193876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3706-0768-4b71-acc8-1d6f8df50fe0 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.194183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3706-0768-4b71-acc8-1d6f8df50fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.195403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3706-0768-4b71-acc8-1d6f8df50fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.247913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-cc9b3706-0768-4b71-acc8-1d6f8df50fe0 None None] MFA Rules not processed for user `a4c425fce3e94daf89d14a220b761a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:02.249503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cc9b3706-0768-4b71-acc8-1d6f8df50fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:02.339758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 938/1854] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:02.345368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228cae8-c7a3-443c-a32d-c72e69359fc3 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.436547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228cae8-c7a3-443c-a32d-c72e69359fc3 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.445973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4253ed9-cd34-4a2f-bea5-33c98c1d176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.446805 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4253ed9-cd34-4a2f-bea5-33c98c1d176c admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.447099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4253ed9-cd34-4a2f-bea5-33c98c1d176c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.447554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4253ed9-cd34-4a2f-bea5-33c98c1d176c admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.448495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4253ed9-cd34-4a2f-bea5-33c98c1d176c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.450832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4253ed9-cd34-4a2f-bea5-33c98c1d176c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.563668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 917/1855] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:02.568818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b3706-0768-4b71-acc8-1d6f8df50fe0 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.577306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b3706-0768-4b71-acc8-1d6f8df50fe0 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.652012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41c85-4ff8-4ad3-9204-a7ba1d9cf338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.652636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41c85-4ff8-4ad3-9204-a7ba1d9cf338 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.652950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41c85-4ff8-4ad3-9204-a7ba1d9cf338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.653226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41c85-4ff8-4ad3-9204-a7ba1d9cf338 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.653892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41c85-4ff8-4ad3-9204-a7ba1d9cf338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.656132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41c85-4ff8-4ad3-9204-a7ba1d9cf338 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.776061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:02.782193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 939/1856] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/users => generated 300 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:02.786570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4253ed9-cd34-4a2f-bea5-33c98c1d176c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.798360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4253ed9-cd34-4a2f-bea5-33c98c1d176c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.803478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b243c-7960-4353-b49b-85d3a63ce04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.804121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b243c-7960-4353-b49b-85d3a63ce04b admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.804400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b243c-7960-4353-b49b-85d3a63ce04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.804675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b243c-7960-4353-b49b-85d3a63ce04b admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.805330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b243c-7960-4353-b49b-85d3a63ce04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.807036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b243c-7960-4353-b49b-85d3a63ce04b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.819716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 918/1857] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:02.823870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41c85-4ff8-4ad3-9204-a7ba1d9cf338 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.835158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41c85-4ff8-4ad3-9204-a7ba1d9cf338 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.839804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a123b-55f6-4eeb-8b4e-cebc1ca9786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.840473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a123b-55f6-4eeb-8b4e-cebc1ca9786b admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.840753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a123b-55f6-4eeb-8b4e-cebc1ca9786b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.841026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a123b-55f6-4eeb-8b4e-cebc1ca9786b admin admin] PATH_INFO: `/v3/projects/51d62544d09e42b7ae721427cf6b74c1/users/9349905100b74ad8b9c1ab256a6fd4f7/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.856768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a123b-55f6-4eeb-8b4e-cebc1ca9786b admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d62544d09e42b7ae721427cf6b74c1, user_id=9349905100b74ad8b9c1ab256a6fd4f7, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.859538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a123b-55f6-4eeb-8b4e-cebc1ca9786b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:03.000424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 940/1858] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:02 2022] PUT /identity/v3/projects/51d62544d09e42b7ae721427cf6b74c1/users/9349905100b74ad8b9c1ab256a6fd4f7/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:03.009302 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b243c-7960-4353-b49b-85d3a63ce04b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:03.018177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b243c-7960-4353-b49b-85d3a63ce04b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:03.045200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:03.103024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd93a-c51c-4970-8e13-f21fecf9b956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:03.104363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd93a-c51c-4970-8e13-f21fecf9b956 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:03.104712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd93a-c51c-4970-8e13-f21fecf9b956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:03.105095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd93a-c51c-4970-8e13-f21fecf9b956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:03.105974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabd93a-c51c-4970-8e13-f21fecf9b956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:03.107994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabd93a-c51c-4970-8e13-f21fecf9b956 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:03.121650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 919/1859] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:03.126563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a123b-55f6-4eeb-8b4e-cebc1ca9786b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:03.142613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a123b-55f6-4eeb-8b4e-cebc1ca9786b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:03.147911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:03.148735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:03.148991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:03.149570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] PATH_INFO: `/v3/projects/51d62544d09e42b7ae721427cf6b74c1/users/9349905100b74ad8b9c1ab256a6fd4f7/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:03.153039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d62544d09e42b7ae721427cf6b74c1, user_id=9349905100b74ad8b9c1ab256a6fd4f7, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:03.156235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:03.168584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 941/1860] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:03 2022] PUT /identity/v3/projects/51d62544d09e42b7ae721427cf6b74c1/users/9349905100b74ad8b9c1ab256a6fd4f7/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:03.180381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bf1e0-129d-4c99-9353-f7a7d00cee2e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:03.180690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bf1e0-129d-4c99-9353-f7a7d00cee2e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:03.180937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bf1e0-129d-4c99-9353-f7a7d00cee2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:03.213999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-c52bf1e0-129d-4c99-9353-f7a7d00cee2e None None] MFA Rules not processed for user `9349905100b74ad8b9c1ab256a6fd4f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:03.215196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c52bf1e0-129d-4c99-9353-f7a7d00cee2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:03.319339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 920/1861] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:03.331486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:03.333606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:03.406585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b67d3-c7fa-412c-b7b1-2716a54e909c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:03.412334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9640-8779-449d-930a-5bdeac09ccac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:03.413123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9640-8779-449d-930a-5bdeac09ccac service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:03.413352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9640-8779-449d-930a-5bdeac09ccac service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:03.413626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9640-8779-449d-930a-5bdeac09ccac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:03.425213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de9640-8779-449d-930a-5bdeac09ccac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:03.426737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de9640-8779-449d-930a-5bdeac09ccac service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:03.443591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 942/1862] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:03.736086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bf1e0-129d-4c99-9353-f7a7d00cee2e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:03.941232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bf1e0-129d-4c99-9353-f7a7d00cee2e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:03.953801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:03.954938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:03.955052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:03.955312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] PATH_INFO: `/v3/users/a4c425fce3e94daf89d14a220b761a7f` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:03.956881 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4c425fce3e94daf89d14a220b761a7f)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:03.958898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:04.200778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:04.203601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4c425fce3e94daf89d14a220b761a7f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:04.311906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4c425fce3e94daf89d14a220b761a7f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:04.312816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4c425fce3e94daf89d14a220b761a7f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:04.320587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4c425fce3e94daf89d14a220b761a7f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:04.321312 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4c425fce3e94daf89d14a220b761a7f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:04.344134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 921/1863] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:03 2022] DELETE /identity/v3/users/a4c425fce3e94daf89d14a220b761a7f => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:04.352152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe178a4-8959-4258-b1cf-f86a8b98f5b3 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:04.352455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe178a4-8959-4258-b1cf-f86a8b98f5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:04.352800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe178a4-8959-4258-b1cf-f86a8b98f5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:04.375467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-afe178a4-8959-4258-b1cf-f86a8b98f5b3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:04.376918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-afe178a4-8959-4258-b1cf-f86a8b98f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:04.489025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 943/1864] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:04.499186 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:04.501236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:04.585383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf9a6-51ed-470c-a7ce-4f192a2947cc admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:04.589329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20117f6-2d21-477e-b145-66a18e25d4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:04.590003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20117f6-2d21-477e-b145-66a18e25d4ce service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:04.590363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20117f6-2d21-477e-b145-66a18e25d4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:04.590616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20117f6-2d21-477e-b145-66a18e25d4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:04.600560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20117f6-2d21-477e-b145-66a18e25d4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:04.602293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20117f6-2d21-477e-b145-66a18e25d4ce service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:04.613563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 922/1865] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:05.354848 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe178a4-8959-4258-b1cf-f86a8b98f5b3 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:05.356942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-afe178a4-8959-4258-b1cf-f86a8b98f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:05.379977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe178a4-8959-4258-b1cf-f86a8b98f5b3 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:05.387504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:05.388332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:05.388492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:05.388738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] PATH_INFO: `/v3/projects/87c6230df86c4e1b96f1bbe34809443f` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:05.391164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c6230df86c4e1b96f1bbe34809443f)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:05.400608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:05.597788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c6230df86c4e1b96f1bbe34809443f', 'request_id': 'req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:05.720361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c6230df86c4e1b96f1bbe34809443f', 'request_id': 'req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:05.732984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c6230df86c4e1b96f1bbe34809443f', 'request_id': 'req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:05.742395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:05.762851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] The token cache is being invalidate because project 87c6230df86c4e1b96f1bbe34809443f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:05.763397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:05.764585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:05.767779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 944/1866] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:05 2022] DELETE /identity/v3/projects/87c6230df86c4e1b96f1bbe34809443f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:05.788703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20117f6-2d21-477e-b145-66a18e25d4ce service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:05.804802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20117f6-2d21-477e-b145-66a18e25d4ce service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:05.883419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:05.883994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:05.884286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:05.884561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] PATH_INFO: `/v3/users/9349905100b74ad8b9c1ab256a6fd4f7` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:05.885906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9349905100b74ad8b9c1ab256a6fd4f7)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:05.888014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:06.254320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9349905100b74ad8b9c1ab256a6fd4f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.380793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9349905100b74ad8b9c1ab256a6fd4f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.381768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9349905100b74ad8b9c1ab256a6fd4f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.407723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9349905100b74ad8b9c1ab256a6fd4f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.408600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b1c6bc55-aa6c-4a8e-ba45-90019dd00a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9349905100b74ad8b9c1ab256a6fd4f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.412440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:06.413231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:06.417495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:06.435687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f243a6f-b2e8-46f7-9f2c-92dca0925f4f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:06.442331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:06.443286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:06.443733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:06.444597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] PATH_INFO: `/v3/users/a43147c973d2413d95aa30792ab0ef16` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:06.446271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a43147c973d2413d95aa30792ab0ef16)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:06.448836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:06.460193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 923/1867] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:05 2022] DELETE /identity/v3/users/9349905100b74ad8b9c1ab256a6fd4f7 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:06.846574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a43147c973d2413d95aa30792ab0ef16'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.869236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a43147c973d2413d95aa30792ab0ef16'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.998359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a43147c973d2413d95aa30792ab0ef16'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.000040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a43147c973d2413d95aa30792ab0ef16'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.000992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a43147c973d2413d95aa30792ab0ef16'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.025216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 945/1868] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:06 2022] DELETE /identity/v3/users/a43147c973d2413d95aa30792ab0ef16 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:07.029984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628106a5-0c7f-440b-a9ec-641634892771 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.030964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628106a5-0c7f-440b-a9ec-641634892771 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.032836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628106a5-0c7f-440b-a9ec-641634892771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.059411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-628106a5-0c7f-440b-a9ec-641634892771 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:07.061543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-628106a5-0c7f-440b-a9ec-641634892771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.143890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 924/1869] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:07.154759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:07.156504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.228888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628106a5-0c7f-440b-a9ec-641634892771 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:07.231735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-628106a5-0c7f-440b-a9ec-641634892771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.250132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5fc08-7d75-404c-88c5-7e967691b572 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:07.256524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ceb4c-2b44-4526-8e8e-081b8dbe5de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:07.257468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ceb4c-2b44-4526-8e8e-081b8dbe5de5 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.257622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ceb4c-2b44-4526-8e8e-081b8dbe5de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.257925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ceb4c-2b44-4526-8e8e-081b8dbe5de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.262186 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628106a5-0c7f-440b-a9ec-641634892771 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:07.273801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ceb4c-2b44-4526-8e8e-081b8dbe5de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:07.277825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ceb4c-2b44-4526-8e8e-081b8dbe5de5 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:07.278088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:07.278088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.278088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.278088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] PATH_INFO: `/v3/projects/51d62544d09e42b7ae721427cf6b74c1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.278936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d62544d09e42b7ae721427cf6b74c1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:07.281059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:07.287753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 946/1870] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:07.422119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d62544d09e42b7ae721427cf6b74c1', 'request_id': 'req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.529721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d62544d09e42b7ae721427cf6b74c1', 'request_id': 'req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.537483 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d62544d09e42b7ae721427cf6b74c1', 'request_id': 'req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.555473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] The token cache is being invalidate because project 51d62544d09e42b7ae721427cf6b74c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:07.556211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.557071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6e5e46dc-8b95-45d0-8f7b-73d788430bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.562515 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 925/1871] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:07 2022] DELETE /identity/v3/projects/51d62544d09e42b7ae721427cf6b74c1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:07.564884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd5243-6295-4562-be4e-268827af8ff8 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.565236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd5243-6295-4562-be4e-268827af8ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.565584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd5243-6295-4562-be4e-268827af8ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.604392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-51dd5243-6295-4562-be4e-268827af8ff8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:07.605616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-51dd5243-6295-4562-be4e-268827af8ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.712872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 947/1872] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:07.722548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c186d6-97e3-4d99-8767-6429c086884b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.722833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c186d6-97e3-4d99-8767-6429c086884b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.723592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c186d6-97e3-4d99-8767-6429c086884b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.749409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f6c186d6-97e3-4d99-8767-6429c086884b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:07.751376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f6c186d6-97e3-4d99-8767-6429c086884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.767531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 926/1873] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:07.772027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd5243-6295-4562-be4e-268827af8ff8 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:07.779765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:07.788620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd5243-6295-4562-be4e-268827af8ff8 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:07.802387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:07.806121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.806531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.806850 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.807708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:07.809117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:07.812334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:07.831115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 948/1874] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:50:07.835747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c186d6-97e3-4d99-8767-6429c086884b None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:07.856916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c186d6-97e3-4d99-8767-6429c086884b None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:07.873317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61c07-746f-486e-b4f8-2fae5a1c9a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:07.874018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61c07-746f-486e-b4f8-2fae5a1c9a38 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.874300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61c07-746f-486e-b4f8-2fae5a1c9a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.874624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61c07-746f-486e-b4f8-2fae5a1c9a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.875266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab61c07-746f-486e-b4f8-2fae5a1c9a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:07.877951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab61c07-746f-486e-b4f8-2fae5a1c9a38 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:07.997879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.000076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:08.018901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd0d87-6325-47e0-a085-03a2f57bca23 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.026983 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 927/1875] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:07 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:08.027534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.028551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.029091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.029200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] PATH_INFO: `/v3/projects/775b89b6f18948d58cb96d4593951158` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.030939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=775b89b6f18948d58cb96d4593951158)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.031331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61c07-746f-486e-b4f8-2fae5a1c9a38 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.034405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.148467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61c07-746f-486e-b4f8-2fae5a1c9a38 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.155585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77b33a-8d83-4fb8-b479-9e4c1d0faa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.156492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77b33a-8d83-4fb8-b479-9e4c1d0faa5e admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.158301 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77b33a-8d83-4fb8-b479-9e4c1d0faa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.158301 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77b33a-8d83-4fb8-b479-9e4c1d0faa5e admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.159257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77b33a-8d83-4fb8-b479-9e4c1d0faa5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.162799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77b33a-8d83-4fb8-b479-9e4c1d0faa5e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.208286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '775b89b6f18948d58cb96d4593951158', 'request_id': 'req-10625988-11c2-48e0-8c18-d2a2f6a5f623'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.258745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:08.271873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 928/1876] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:08 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:08.273197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77b33a-8d83-4fb8-b479-9e4c1d0faa5e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.297501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77b33a-8d83-4fb8-b479-9e4c1d0faa5e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.332294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '775b89b6f18948d58cb96d4593951158', 'request_id': 'req-10625988-11c2-48e0-8c18-d2a2f6a5f623'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.343084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '775b89b6f18948d58cb96d4593951158', 'request_id': 'req-10625988-11c2-48e0-8c18-d2a2f6a5f623'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.368307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] The token cache is being invalidate because project 775b89b6f18948d58cb96d4593951158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:08.368891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.370032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.371956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 949/1877] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:07 2022] DELETE /identity/v3/projects/775b89b6f18948d58cb96d4593951158 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:08.377618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.379094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:08.382359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.382990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.383270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.383552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.384310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.385958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.395184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10625988-11c2-48e0-8c18-d2a2f6a5f623 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.398365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 929/1878] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:08.402517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.402742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.406551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.406551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.406551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] PATH_INFO: `/v3/users/b4584607b4544893984c0aa3e20fef43` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.406551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4584607b4544893984c0aa3e20fef43)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.406935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:08.407409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.429444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92340e-e04c-4bcb-af95-a25187988a9e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.440216 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e1129-3c51-46ac-9587-b01db601168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.441271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e1129-3c51-46ac-9587-b01db601168b admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.441778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e1129-3c51-46ac-9587-b01db601168b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.442318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e1129-3c51-46ac-9587-b01db601168b admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.443520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e1129-3c51-46ac-9587-b01db601168b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.446726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e1129-3c51-46ac-9587-b01db601168b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.469462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 930/1879] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:08.473997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e1129-3c51-46ac-9587-b01db601168b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.484514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e1129-3c51-46ac-9587-b01db601168b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.489693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081a171-e982-4c57-ab2c-6a083a2e4708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.490571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081a171-e982-4c57-ab2c-6a083a2e4708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.490856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081a171-e982-4c57-ab2c-6a083a2e4708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.491136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081a171-e982-4c57-ab2c-6a083a2e4708 admin admin] PATH_INFO: `/v3/projects/cd8360160397475584b37514525cd1e0/users/138824d726f646fc9598e56d6098aa1b/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.510853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081a171-e982-4c57-ab2c-6a083a2e4708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8360160397475584b37514525cd1e0, user_id=138824d726f646fc9598e56d6098aa1b, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.513416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081a171-e982-4c57-ab2c-6a083a2e4708 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.673010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 931/1880] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:08 2022] PUT /identity/v3/projects/cd8360160397475584b37514525cd1e0/users/138824d726f646fc9598e56d6098aa1b/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:08.684097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1e2f-6cd8-4ca6-b9b5-0ddbf757ab6f None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.684399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1e2f-6cd8-4ca6-b9b5-0ddbf757ab6f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.684676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1e2f-6cd8-4ca6-b9b5-0ddbf757ab6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.726797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7f4e1e2f-6cd8-4ca6-b9b5-0ddbf757ab6f None None] MFA Rules not processed for user `138824d726f646fc9598e56d6098aa1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:08.728648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7f4e1e2f-6cd8-4ca6-b9b5-0ddbf757ab6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:08.765116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4584607b4544893984c0aa3e20fef43'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.782191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4584607b4544893984c0aa3e20fef43'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.810940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 932/1881] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:08.816874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1e2f-6cd8-4ca6-b9b5-0ddbf757ab6f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.860772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4584607b4544893984c0aa3e20fef43'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.861509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4584607b4544893984c0aa3e20fef43'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.862255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4584607b4544893984c0aa3e20fef43'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.888269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1e2f-6cd8-4ca6-b9b5-0ddbf757ab6f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.898183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccccc8-e90b-428c-b2fd-e1f6ac6d76d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.898290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccccc8-e90b-428c-b2fd-e1f6ac6d76d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.899456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccccc8-e90b-428c-b2fd-e1f6ac6d76d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.899456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccccc8-e90b-428c-b2fd-e1f6ac6d76d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.899769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccccc8-e90b-428c-b2fd-e1f6ac6d76d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.906116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccccc8-e90b-428c-b2fd-e1f6ac6d76d0 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.906359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 950/1882] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:08 2022] DELETE /identity/v3/users/b4584607b4544893984c0aa3e20fef43 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:09.037561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 933/1883] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:08 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:09.038331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.038331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:09.151731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc0b3e-d111-4659-ad9b-9e3678c8bb27 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.267099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c31b5a-0dba-49aa-8ca6-80e840b92abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.267099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c31b5a-0dba-49aa-8ca6-80e840b92abe admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.267099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c31b5a-0dba-49aa-8ca6-80e840b92abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.267099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c31b5a-0dba-49aa-8ca6-80e840b92abe admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.267474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c31b5a-0dba-49aa-8ca6-80e840b92abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.269825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c31b5a-0dba-49aa-8ca6-80e840b92abe admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.380829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 951/1884] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/users => generated 297 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:09.385775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccccc8-e90b-428c-b2fd-e1f6ac6d76d0 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.399710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccccc8-e90b-428c-b2fd-e1f6ac6d76d0 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.405109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.406223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.407778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.407778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.408562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.410297 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.462019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 934/1885] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:09.465900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c31b5a-0dba-49aa-8ca6-80e840b92abe admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.481532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c31b5a-0dba-49aa-8ca6-80e840b92abe admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.489177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158db9d-dbed-441f-9967-5c213cda119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.490318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158db9d-dbed-441f-9967-5c213cda119b admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.490864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158db9d-dbed-441f-9967-5c213cda119b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.491330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158db9d-dbed-441f-9967-5c213cda119b admin admin] PATH_INFO: `/v3/projects/a0f90864019b4332b3c95e93d54d2037/users/e28819354cc441eabea68a060adec0fe/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.510353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158db9d-dbed-441f-9967-5c213cda119b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f90864019b4332b3c95e93d54d2037, user_id=e28819354cc441eabea68a060adec0fe, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.513385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158db9d-dbed-441f-9967-5c213cda119b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.582908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.584880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:09.603383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28214b0d-5595-4160-99c9-7e185a2f2432 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.615163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.615943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.616425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.616853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] PATH_INFO: `/v3/projects/b113a0e20804437a8f18a67b0d2a9b13` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.620540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 952/1886] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:09 2022] PUT /identity/v3/projects/a0f90864019b4332b3c95e93d54d2037/users/e28819354cc441eabea68a060adec0fe/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:09.621338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b113a0e20804437a8f18a67b0d2a9b13)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.623352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.625054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158db9d-dbed-441f-9967-5c213cda119b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.706075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158db9d-dbed-441f-9967-5c213cda119b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.711243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f8056-c4a5-45a1-b13c-9e576adfc662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.711870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f8056-c4a5-45a1-b13c-9e576adfc662 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.712148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f8056-c4a5-45a1-b13c-9e576adfc662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.712426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f8056-c4a5-45a1-b13c-9e576adfc662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.713084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f8056-c4a5-45a1-b13c-9e576adfc662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.714735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f8056-c4a5-45a1-b13c-9e576adfc662 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.726263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 953/1887] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:09.730516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f8056-c4a5-45a1-b13c-9e576adfc662 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.742576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f8056-c4a5-45a1-b13c-9e576adfc662 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.747611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a93e6-0325-409c-896b-21d7a48ba51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.748288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a93e6-0325-409c-896b-21d7a48ba51c admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.748565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a93e6-0325-409c-896b-21d7a48ba51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.748838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a93e6-0325-409c-896b-21d7a48ba51c admin admin] PATH_INFO: `/v3/projects/a0f90864019b4332b3c95e93d54d2037/users/e28819354cc441eabea68a060adec0fe/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.751856 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a93e6-0325-409c-896b-21d7a48ba51c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f90864019b4332b3c95e93d54d2037, user_id=e28819354cc441eabea68a060adec0fe, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.754597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a93e6-0325-409c-896b-21d7a48ba51c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.768830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 954/1888] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:09 2022] PUT /identity/v3/projects/a0f90864019b4332b3c95e93d54d2037/users/e28819354cc441eabea68a060adec0fe/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:09.781437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f81b4-98e1-4979-afd9-18451c10f543 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.781952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f81b4-98e1-4979-afd9-18451c10f543 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.782360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f81b4-98e1-4979-afd9-18451c10f543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.815508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3e8f81b4-98e1-4979-afd9-18451c10f543 None None] MFA Rules not processed for user `e28819354cc441eabea68a060adec0fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:09.816864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3e8f81b4-98e1-4979-afd9-18451c10f543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:09.829754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b113a0e20804437a8f18a67b0d2a9b13', 'request_id': 'req-b51ce47c-7fb5-4c7f-9125-d33565320f97'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.897022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 955/1889] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:09.909433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f81b4-98e1-4979-afd9-18451c10f543 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.910980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3e8f81b4-98e1-4979-afd9-18451c10f543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:09.912378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b113a0e20804437a8f18a67b0d2a9b13', 'request_id': 'req-b51ce47c-7fb5-4c7f-9125-d33565320f97'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.920178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b113a0e20804437a8f18a67b0d2a9b13', 'request_id': 'req-b51ce47c-7fb5-4c7f-9125-d33565320f97'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.926562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:09.933816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] The token cache is being invalidate because project b113a0e20804437a8f18a67b0d2a9b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:09.934141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.935030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.941433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 935/1890] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:09 2022] DELETE /identity/v3/projects/b113a0e20804437a8f18a67b0d2a9b13 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:09.944744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.946890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:09.982236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f81b4-98e1-4979-afd9-18451c10f543 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.987461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.988077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.988422 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.988698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.990639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:10.005255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.006779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.017913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 956/1891] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:10.026664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ce47c-7fb5-4c7f-9125-d33565320f97 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:10.032714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:10.033366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.033649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.033958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] PATH_INFO: `/v3/users/5542183c6cbc40259c1c105bf02c5c82` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.035169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5542183c6cbc40259c1c105bf02c5c82)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.037092 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.262438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:10.272170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5542183c6cbc40259c1c105bf02c5c82'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.340267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5542183c6cbc40259c1c105bf02c5c82'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.340996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5542183c6cbc40259c1c105bf02c5c82'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.350221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5542183c6cbc40259c1c105bf02c5c82'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.351314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5542183c6cbc40259c1c105bf02c5c82'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.402817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 936/1892] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:09 2022] DELETE /identity/v3/users/5542183c6cbc40259c1c105bf02c5c82 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:10.758353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:10.761023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:10.799112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7243f-1d6a-45af-9019-2c4eba099347 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:10.869884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:10.870457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.870737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.871018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] PATH_INFO: `/v3/users/138824d726f646fc9598e56d6098aa1b` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.872230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138824d726f646fc9598e56d6098aa1b)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.874282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.987303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:10.999548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.011780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.026837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:11.027689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138824d726f646fc9598e56d6098aa1b'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.048736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138824d726f646fc9598e56d6098aa1b'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.074068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30c06e-faa7-40ae-9a3f-71f78e5d89e8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.081380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.081969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.082256 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.082532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] PATH_INFO: `/v3/projects/ab0909c833f144fcb5b6ab5faa515ff5` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.084023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0909c833f144fcb5b6ab5faa515ff5)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.086201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.134475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138824d726f646fc9598e56d6098aa1b'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.135103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138824d726f646fc9598e56d6098aa1b'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.135809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8d1a2ba8-c596-4a00-ab57-d0ce202738c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138824d726f646fc9598e56d6098aa1b'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.155131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 957/1893] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:10 2022] DELETE /identity/v3/users/138824d726f646fc9598e56d6098aa1b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:11.160896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91025ac4-532a-4179-882c-4b145e35b06c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.161335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91025ac4-532a-4179-882c-4b145e35b06c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.161696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91025ac4-532a-4179-882c-4b145e35b06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.183819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-91025ac4-532a-4179-882c-4b145e35b06c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:11.185126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-91025ac4-532a-4179-882c-4b145e35b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.209009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0909c833f144fcb5b6ab5faa515ff5', 'request_id': 'req-85a87f28-379f-4c30-9490-72f644035180'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.256580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 958/1894] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:11.266904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91025ac4-532a-4179-882c-4b145e35b06c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.268295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-91025ac4-532a-4179-882c-4b145e35b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.271464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0909c833f144fcb5b6ab5faa515ff5', 'request_id': 'req-85a87f28-379f-4c30-9490-72f644035180'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.278848 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0909c833f144fcb5b6ab5faa515ff5', 'request_id': 'req-85a87f28-379f-4c30-9490-72f644035180'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.296466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] The token cache is being invalidate because project ab0909c833f144fcb5b6ab5faa515ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:11.296764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.297688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-85a87f28-379f-4c30-9490-72f644035180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.302202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 937/1895] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:10 2022] DELETE /identity/v3/projects/ab0909c833f144fcb5b6ab5faa515ff5 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:11.305778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349c979-f7dc-47e2-8642-cf9c1876c551 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.306113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349c979-f7dc-47e2-8642-cf9c1876c551 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.306405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349c979-f7dc-47e2-8642-cf9c1876c551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.332002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b349c979-f7dc-47e2-8642-cf9c1876c551 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:11.334247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91025ac4-532a-4179-882c-4b145e35b06c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.334398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b349c979-f7dc-47e2-8642-cf9c1876c551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.335864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:11.396384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.396779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.397250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.397540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.399634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.480083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 938/1896] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:11.484207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859d79a-5e86-4237-b50d-0e64bbe163be None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.484560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859d79a-5e86-4237-b50d-0e64bbe163be None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.484803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859d79a-5e86-4237-b50d-0e64bbe163be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.538107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-1859d79a-5e86-4237-b50d-0e64bbe163be None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:11.541205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1859d79a-5e86-4237-b50d-0e64bbe163be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.566563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 939/1897] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:11.569335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859d79a-5e86-4237-b50d-0e64bbe163be None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.582782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859d79a-5e86-4237-b50d-0e64bbe163be None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.584600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.586122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.589781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.590991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.591458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.592057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.592949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:11.593439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.596478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.598937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 959/1898] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:11.612787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:11.624624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 940/1899] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:11.627561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.649073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61a454-8861-4516-83ef-4f8bdd5072ae service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.663218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f61196-babf-45a5-a660-d347fe6a396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.664574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f61196-babf-45a5-a660-d347fe6a396d admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.665259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f61196-babf-45a5-a660-d347fe6a396d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.665947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f61196-babf-45a5-a660-d347fe6a396d admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.667984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f61196-babf-45a5-a660-d347fe6a396d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.673050 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f61196-babf-45a5-a660-d347fe6a396d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.827464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 960/1900] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:11.833350 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.850882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ecb1b-d5bc-4bfe-89b4-daa472d1a3ec admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.986809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc8978-692c-406d-b25a-033be1893749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.987535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc8978-692c-406d-b25a-033be1893749 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.988206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc8978-692c-406d-b25a-033be1893749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.988743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc8978-692c-406d-b25a-033be1893749 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.989713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc8978-692c-406d-b25a-033be1893749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.991794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc8978-692c-406d-b25a-033be1893749 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.100019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:12.117040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 941/1901] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/users => generated 291 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:12.118917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f61196-babf-45a5-a660-d347fe6a396d admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.143417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f61196-babf-45a5-a660-d347fe6a396d admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.162283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65403ad1-4621-4fed-a9d2-9de086e60a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.162283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65403ad1-4621-4fed-a9d2-9de086e60a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.162283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65403ad1-4621-4fed-a9d2-9de086e60a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.162283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65403ad1-4621-4fed-a9d2-9de086e60a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.162554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65403ad1-4621-4fed-a9d2-9de086e60a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.162554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65403ad1-4621-4fed-a9d2-9de086e60a64 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.176309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 961/1902] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:12.188685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc8978-692c-406d-b25a-033be1893749 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.202825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc8978-692c-406d-b25a-033be1893749 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.214361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.215071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.215443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.215750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.216415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.218184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.232897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 942/1903] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:12.238618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65403ad1-4621-4fed-a9d2-9de086e60a64 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.251567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65403ad1-4621-4fed-a9d2-9de086e60a64 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.257908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de4039-3604-4bd5-9225-404bce652dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.258671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de4039-3604-4bd5-9225-404bce652dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.258962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de4039-3604-4bd5-9225-404bce652dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.259422 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de4039-3604-4bd5-9225-404bce652dcc admin admin] PATH_INFO: `/v3/projects/c733cf78a3874be595007784100c4f39/users/853a84096a6f475f9860d3999ae3d6c3/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.279707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de4039-3604-4bd5-9225-404bce652dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c733cf78a3874be595007784100c4f39, user_id=853a84096a6f475f9860d3999ae3d6c3, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.283385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de4039-3604-4bd5-9225-404bce652dcc admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.315036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.318513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.335181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c22668-e9c2-488b-85db-2f1ae0a7eb82 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.348809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.350529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.350816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.351100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] PATH_INFO: `/v3/projects/cd8360160397475584b37514525cd1e0` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.352374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8360160397475584b37514525cd1e0)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.354421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.397448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 962/1904] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:12 2022] PUT /identity/v3/projects/c733cf78a3874be595007784100c4f39/users/853a84096a6f475f9860d3999ae3d6c3/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:12.410852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7f7cc-24b3-41e9-b817-ed71616febc5 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.411137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7f7cc-24b3-41e9-b817-ed71616febc5 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.411411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7f7cc-24b3-41e9-b817-ed71616febc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.445844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a8c7f7cc-24b3-41e9-b817-ed71616febc5 None None] MFA Rules not processed for user `853a84096a6f475f9860d3999ae3d6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:12.447302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a8c7f7cc-24b3-41e9-b817-ed71616febc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.464510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8360160397475584b37514525cd1e0', 'request_id': 'req-55c84ac8-37ee-49e0-8085-907526274ad0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:12.525518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 963/1905] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:12.531306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7f7cc-24b3-41e9-b817-ed71616febc5 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.547686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8360160397475584b37514525cd1e0', 'request_id': 'req-55c84ac8-37ee-49e0-8085-907526274ad0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:12.564158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8360160397475584b37514525cd1e0', 'request_id': 'req-55c84ac8-37ee-49e0-8085-907526274ad0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:12.583572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] The token cache is being invalidate because project cd8360160397475584b37514525cd1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:12.583871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:12.584794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:12.586615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 943/1906] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:12 2022] DELETE /identity/v3/projects/cd8360160397475584b37514525cd1e0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:12.591806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.598405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.636942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7f7cc-24b3-41e9-b817-ed71616febc5 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.643017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.644013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.644401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.644751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.646278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.650179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.697167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c84ac8-37ee-49e0-8085-907526274ad0 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.702170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.702907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.704201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.704201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] PATH_INFO: `/v3/users/e28819354cc441eabea68a060adec0fe` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.706053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e28819354cc441eabea68a060adec0fe)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.708126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.710033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:12.806260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 964/1907] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:12 2022] POST /identity/v3/projects => generated 381 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:12.807838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.811932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.850718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:12.947583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf6f46-2c6e-4efe-a3f8-804da3b7c253 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.954765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e77cd3-997f-4975-b24b-16a38666dde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.955497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e77cd3-997f-4975-b24b-16a38666dde8 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.955785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e77cd3-997f-4975-b24b-16a38666dde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.956062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e77cd3-997f-4975-b24b-16a38666dde8 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.956742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e77cd3-997f-4975-b24b-16a38666dde8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.958920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e77cd3-997f-4975-b24b-16a38666dde8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.025798 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e28819354cc441eabea68a060adec0fe'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.085819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 965/1908] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:12 2022] POST /identity/v3/users => generated 297 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:13.089151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e77cd3-997f-4975-b24b-16a38666dde8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.102617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e77cd3-997f-4975-b24b-16a38666dde8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.108047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c60f9f-9722-4983-9734-dcc1b1c6eed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.108970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c60f9f-9722-4983-9734-dcc1b1c6eed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.109462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c60f9f-9722-4983-9734-dcc1b1c6eed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.110603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c60f9f-9722-4983-9734-dcc1b1c6eed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.111356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c60f9f-9722-4983-9734-dcc1b1c6eed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.112997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c60f9f-9722-4983-9734-dcc1b1c6eed4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.124832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 966/1909] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:13.129540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c60f9f-9722-4983-9734-dcc1b1c6eed4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.134500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e28819354cc441eabea68a060adec0fe'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.136132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e28819354cc441eabea68a060adec0fe'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.142568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c60f9f-9722-4983-9734-dcc1b1c6eed4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.153270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a698b-3e01-4145-85c8-2f844ee96d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.154076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a698b-3e01-4145-85c8-2f844ee96d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.154686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a698b-3e01-4145-85c8-2f844ee96d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.154977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a698b-3e01-4145-85c8-2f844ee96d2b admin admin] PATH_INFO: `/v3/projects/520b994e84594b81a9dc40600f54f244/users/f38a5876e5ec46fb9758874b13d04976/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.157379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e28819354cc441eabea68a060adec0fe'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.158560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e28819354cc441eabea68a060adec0fe'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.188921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a698b-3e01-4145-85c8-2f844ee96d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=520b994e84594b81a9dc40600f54f244, user_id=f38a5876e5ec46fb9758874b13d04976, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.212023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a698b-3e01-4145-85c8-2f844ee96d2b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.224225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 944/1910] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:12 2022] DELETE /identity/v3/users/e28819354cc441eabea68a060adec0fe => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:13.334943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 967/1911] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:13 2022] PUT /identity/v3/projects/520b994e84594b81a9dc40600f54f244/users/f38a5876e5ec46fb9758874b13d04976/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:13.336553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.338953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:13.470796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa897a2-9c0a-44c7-9be5-4e99079e168b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.584348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006505e7-453f-4e20-8c8e-4917c9a659c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.584348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006505e7-453f-4e20-8c8e-4917c9a659c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.584348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006505e7-453f-4e20-8c8e-4917c9a659c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.584348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006505e7-453f-4e20-8c8e-4917c9a659c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.584806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006505e7-453f-4e20-8c8e-4917c9a659c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.585377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006505e7-453f-4e20-8c8e-4917c9a659c3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.619349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 945/1912] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:13.623945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a698b-3e01-4145-85c8-2f844ee96d2b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.636867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a698b-3e01-4145-85c8-2f844ee96d2b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.642652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.643401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.643688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.643955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] PATH_INFO: `/v3/projects/520b994e84594b81a9dc40600f54f244/users/f38a5876e5ec46fb9758874b13d04976/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.646052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=520b994e84594b81a9dc40600f54f244, user_id=f38a5876e5ec46fb9758874b13d04976, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.650885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.673192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 968/1913] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:13 2022] PUT /identity/v3/projects/520b994e84594b81a9dc40600f54f244/users/f38a5876e5ec46fb9758874b13d04976/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:13.680956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6c94-109d-49d2-816e-a352d61bff28 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.681284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6c94-109d-49d2-816e-a352d61bff28 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.682238 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6c94-109d-49d2-816e-a352d61bff28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.743603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a57a6c94-109d-49d2-816e-a352d61bff28 None None] MFA Rules not processed for user `f38a5876e5ec46fb9758874b13d04976`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:13.747402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a57a6c94-109d-49d2-816e-a352d61bff28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:13.775438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:13.880040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 946/1914] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:13.894477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.894477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:13.934156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a6c94-109d-49d2-816e-a352d61bff28 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.941199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a57a6c94-109d-49d2-816e-a352d61bff28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:13.978063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcac3a-64f7-4b8c-991f-d3fdff078557 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.987155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6273f2f-7881-441e-8acc-ab3336c27f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.988159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6273f2f-7881-441e-8acc-ab3336c27f68 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.988770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6273f2f-7881-441e-8acc-ab3336c27f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.989255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6273f2f-7881-441e-8acc-ab3336c27f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.009430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6273f2f-7881-441e-8acc-ab3336c27f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.012964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6273f2f-7881-441e-8acc-ab3336c27f68 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.032788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 969/1915] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:14.049781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a6c94-109d-49d2-816e-a352d61bff28 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.061103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.061904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.062267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.062719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] PATH_INFO: `/v3/projects/a0f90864019b4332b3c95e93d54d2037` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.064033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f90864019b4332b3c95e93d54d2037)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.066388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.168460 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6273f2f-7881-441e-8acc-ab3336c27f68 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.182185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f90864019b4332b3c95e93d54d2037', 'request_id': 'req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.184573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6273f2f-7881-441e-8acc-ab3336c27f68 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.257383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76714ab8-c8ca-464a-bc32-75d5f1c08755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.258044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76714ab8-c8ca-464a-bc32-75d5f1c08755 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.258511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76714ab8-c8ca-464a-bc32-75d5f1c08755 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.258701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76714ab8-c8ca-464a-bc32-75d5f1c08755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.260141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f90864019b4332b3c95e93d54d2037', 'request_id': 'req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.260672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-76714ab8-c8ca-464a-bc32-75d5f1c08755 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.269909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f90864019b4332b3c95e93d54d2037', 'request_id': 'req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.274471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:14.287241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] The token cache is being invalidate because project a0f90864019b4332b3c95e93d54d2037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:14.287570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.288659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-84f83bac-9ee3-4e4b-83e6-cb918cdb1c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.290468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 947/1916] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:13 2022] DELETE /identity/v3/projects/a0f90864019b4332b3c95e93d54d2037 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:14.295450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc3d73-d552-4dec-aa9a-38cfcfb81179 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.295729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc3d73-d552-4dec-aa9a-38cfcfb81179 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.296009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc3d73-d552-4dec-aa9a-38cfcfb81179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.325271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9afc3d73-d552-4dec-aa9a-38cfcfb81179 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:14.327136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9afc3d73-d552-4dec-aa9a-38cfcfb81179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.341780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76714ab8-c8ca-464a-bc32-75d5f1c08755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.344572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76714ab8-c8ca-464a-bc32-75d5f1c08755 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.357371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 970/1917] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:14.388513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:14.425747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 948/1918] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:14.434272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2699a-40ab-4ff9-8bf2-965c370620cd None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.434565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2699a-40ab-4ff9-8bf2-965c370620cd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.434938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2699a-40ab-4ff9-8bf2-965c370620cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.461639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-58a2699a-40ab-4ff9-8bf2-965c370620cd None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:14.463754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-58a2699a-40ab-4ff9-8bf2-965c370620cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.472614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 971/1919] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:14.477382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc3d73-d552-4dec-aa9a-38cfcfb81179 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.489536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc3d73-d552-4dec-aa9a-38cfcfb81179 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.494916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.495646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.496053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.496409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.497463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:14.497791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.499834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.523540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 949/1920] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:14.528670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2699a-40ab-4ff9-8bf2-965c370620cd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.540176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2699a-40ab-4ff9-8bf2-965c370620cd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.545428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73ff2a-d701-4725-8978-92d56158f900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.546141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ff2a-d701-4725-8978-92d56158f900 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.546492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ff2a-d701-4725-8978-92d56158f900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.546921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ff2a-d701-4725-8978-92d56158f900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.548270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73ff2a-d701-4725-8978-92d56158f900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.550414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73ff2a-d701-4725-8978-92d56158f900 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.647378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 972/1921] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:14 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:14.652132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.664374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f6943-ea3d-4796-9ecc-f1e5ec40a863 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.755192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78868f80-5a9a-4164-aeea-4a7beede8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.756232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78868f80-5a9a-4164-aeea-4a7beede8bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.756738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78868f80-5a9a-4164-aeea-4a7beede8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.757234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78868f80-5a9a-4164-aeea-4a7beede8bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.758436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78868f80-5a9a-4164-aeea-4a7beede8bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.762017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78868f80-5a9a-4164-aeea-4a7beede8bf2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.872130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 950/1922] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:14 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:14.876992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73ff2a-d701-4725-8978-92d56158f900 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.891161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73ff2a-d701-4725-8978-92d56158f900 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.897011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf7615-a11e-4abf-9e15-42c60344d5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.898587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7615-a11e-4abf-9e15-42c60344d5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.899052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7615-a11e-4abf-9e15-42c60344d5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.899326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7615-a11e-4abf-9e15-42c60344d5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.900025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf7615-a11e-4abf-9e15-42c60344d5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.901637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf7615-a11e-4abf-9e15-42c60344d5a7 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.921633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 973/1923] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:14.927004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78868f80-5a9a-4164-aeea-4a7beede8bf2 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.944769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78868f80-5a9a-4164-aeea-4a7beede8bf2 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.944769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f5551-c097-4aac-9290-86b32e0e5b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.945298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f5551-c097-4aac-9290-86b32e0e5b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.945561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f5551-c097-4aac-9290-86b32e0e5b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.945736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f5551-c097-4aac-9290-86b32e0e5b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.946447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f5551-c097-4aac-9290-86b32e0e5b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.948129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f5551-c097-4aac-9290-86b32e0e5b7a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.962361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 951/1924] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:14.966454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf7615-a11e-4abf-9e15-42c60344d5a7 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.978041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf7615-a11e-4abf-9e15-42c60344d5a7 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.982779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.983391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.983673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.983966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] PATH_INFO: `/v3/projects/8705d2ef80054da8b1257714d6af3907/users/4adeaeb2f3ff4ad0aa5a853badde0c78/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.003049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8705d2ef80054da8b1257714d6af3907, user_id=4adeaeb2f3ff4ad0aa5a853badde0c78, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:15.005922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:15.095167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 974/1925] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:14 2022] PUT /identity/v3/projects/8705d2ef80054da8b1257714d6af3907/users/4adeaeb2f3ff4ad0aa5a853badde0c78/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:15.107152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdc345-f7fc-42c1-babe-4d60d18acc9a None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:15.107423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdc345-f7fc-42c1-babe-4d60d18acc9a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:15.107708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdc345-f7fc-42c1-babe-4d60d18acc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.140141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9cbdc345-f7fc-42c1-babe-4d60d18acc9a None None] MFA Rules not processed for user `4adeaeb2f3ff4ad0aa5a853badde0c78`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:15.141471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9cbdc345-f7fc-42c1-babe-4d60d18acc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:15.228285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 952/1926] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:15.239806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:15.241667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:15.324117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba8a25-7f88-4683-9034-5faccae084ad admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:15.327947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130e864-cfc7-4a5a-97ee-c1327fe5bd0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d874536170494496769b4f37234b2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:15.328663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e864-cfc7-4a5a-97ee-c1327fe5bd0f service glance] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:15.328979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e864-cfc7-4a5a-97ee-c1327fe5bd0f service glance] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:15.329256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e864-cfc7-4a5a-97ee-c1327fe5bd0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.338361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e864-cfc7-4a5a-97ee-c1327fe5bd0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:15.339794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e864-cfc7-4a5a-97ee-c1327fe5bd0f service glance] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:15.349656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 975/1927] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:20.555876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdc345-f7fc-42c1-babe-4d60d18acc9a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:20.558768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9cbdc345-f7fc-42c1-babe-4d60d18acc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:20.598123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:20.644130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdc345-f7fc-42c1-babe-4d60d18acc9a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:20.653311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:20.654528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:20.654954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:20.655332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] PATH_INFO: `/v3/users/c403594e98cf4dcf950ac13172d99f53` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:20.658855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c403594e98cf4dcf950ac13172d99f53)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:20.662179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:20.896888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c403594e98cf4dcf950ac13172d99f53'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:21.020076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c403594e98cf4dcf950ac13172d99f53'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:21.020819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c403594e98cf4dcf950ac13172d99f53'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:21.030188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c403594e98cf4dcf950ac13172d99f53'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:21.030972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c403594e98cf4dcf950ac13172d99f53'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:21.052947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 953/1928] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:20 2022] DELETE /identity/v3/users/c403594e98cf4dcf950ac13172d99f53 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:21.057930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b457735-ce14-46d1-a46e-28edec77a1aa None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:21.058788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b457735-ce14-46d1-a46e-28edec77a1aa None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:21.058928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b457735-ce14-46d1-a46e-28edec77a1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:21.080037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-6b457735-ce14-46d1-a46e-28edec77a1aa None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:21.081297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6b457735-ce14-46d1-a46e-28edec77a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:21.117620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:21.155448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 976/1929] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:21.167847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:21.170258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:21.238888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1489-9232-43f6-954b-50e88ca24b4f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:21.242775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:21.243385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:21.243671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:21.243946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:21.253588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:21.255053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:21.264736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 954/1930] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:21.828961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b457735-ce14-46d1-a46e-28edec77a1aa None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:21.830566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6b457735-ce14-46d1-a46e-28edec77a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:21.851048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b457735-ce14-46d1-a46e-28edec77a1aa None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:21.859834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:21.861525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:21.861964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:21.862103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] PATH_INFO: `/v3/projects/55eab4958382416681cd76e5a6a9b0c0` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:21.867326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55eab4958382416681cd76e5a6a9b0c0)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:21.867326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:21.973286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:21.975126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:21.988772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ac3c5-afac-48a7-8d1d-a085ef0e0c11 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:21.993788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55eab4958382416681cd76e5a6a9b0c0', 'request_id': 'req-6a9761a7-5516-4415-a84f-8ecbb3901e59'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.048105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:22.056175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:22.056847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:22.057124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:22.057429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] PATH_INFO: `/v3/users/853a84096a6f475f9860d3999ae3d6c3` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:22.058670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=853a84096a6f475f9860d3999ae3d6c3)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:22.060606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:22.111547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55eab4958382416681cd76e5a6a9b0c0', 'request_id': 'req-6a9761a7-5516-4415-a84f-8ecbb3901e59'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.118646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55eab4958382416681cd76e5a6a9b0c0', 'request_id': 'req-6a9761a7-5516-4415-a84f-8ecbb3901e59'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.129372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] The token cache is being invalidate because project 55eab4958382416681cd76e5a6a9b0c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:22.129660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.130341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.131921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 977/1931] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:21 2022] DELETE /identity/v3/projects/55eab4958382416681cd76e5a6a9b0c0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:22.135848 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:22.137159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:22.149754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9761a7-5516-4415-a84f-8ecbb3901e59 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:22.154775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:22.155425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:22.155707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:22.156512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] PATH_INFO: `/v3/users/01fe0b3cf1384a1a86b30ea41a5e15a4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:22.157776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01fe0b3cf1384a1a86b30ea41a5e15a4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:22.159717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:22.228266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853a84096a6f475f9860d3999ae3d6c3'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.288526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853a84096a6f475f9860d3999ae3d6c3'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.289299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853a84096a6f475f9860d3999ae3d6c3'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.300356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853a84096a6f475f9860d3999ae3d6c3'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.301616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2ed9486c-c374-430a-a9b8-83e825488c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853a84096a6f475f9860d3999ae3d6c3'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.320878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 955/1932] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:21 2022] DELETE /identity/v3/users/853a84096a6f475f9860d3999ae3d6c3 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:22.325818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edfb3b-666b-424f-8f5b-d72c9833d835 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:22.326138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edfb3b-666b-424f-8f5b-d72c9833d835 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:22.326262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01fe0b3cf1384a1a86b30ea41a5e15a4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.326414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edfb3b-666b-424f-8f5b-d72c9833d835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:22.339570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01fe0b3cf1384a1a86b30ea41a5e15a4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.347154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f2edfb3b-666b-424f-8f5b-d72c9833d835 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:22.348373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f2edfb3b-666b-424f-8f5b-d72c9833d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:22.409491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 956/1933] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:22.422083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edfb3b-666b-424f-8f5b-d72c9833d835 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:22.423831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f2edfb3b-666b-424f-8f5b-d72c9833d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:22.425507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01fe0b3cf1384a1a86b30ea41a5e15a4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.426298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01fe0b3cf1384a1a86b30ea41a5e15a4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.426976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01fe0b3cf1384a1a86b30ea41a5e15a4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:22.441953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 978/1934] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:22 2022] DELETE /identity/v3/users/01fe0b3cf1384a1a86b30ea41a5e15a4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:22.492862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edfb3b-666b-424f-8f5b-d72c9833d835 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:22.527567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:22.552053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:22.552672 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:22.552955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:22.553228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:22.555048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:22.621078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:22.622546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:22.633394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 957/1935] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:23.084801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.088657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:23.100911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30953c-483a-4eea-9c99-783855df0955 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.106327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.107121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.107483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.107832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] PATH_INFO: `/v3/projects/8678c6cc8897418fb26532ec74186e15` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.109355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8678c6cc8897418fb26532ec74186e15)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.111591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.142010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:23.207839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.209943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:23.225282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07a0c6-7e3b-4e4d-b0d2-14a31993971e service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.241937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8678c6cc8897418fb26532ec74186e15', 'request_id': 'req-d337e696-bc10-4635-b405-aea397bf7edf'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.292950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.293642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.293897 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.294180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] PATH_INFO: `/v3/projects/c733cf78a3874be595007784100c4f39` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.295321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c733cf78a3874be595007784100c4f39)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.297357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.311167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8678c6cc8897418fb26532ec74186e15', 'request_id': 'req-d337e696-bc10-4635-b405-aea397bf7edf'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.318457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8678c6cc8897418fb26532ec74186e15', 'request_id': 'req-d337e696-bc10-4635-b405-aea397bf7edf'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.330364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] The token cache is being invalidate because project 8678c6cc8897418fb26532ec74186e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:23.330662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.331323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d337e696-bc10-4635-b405-aea397bf7edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.333225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 979/1936] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:23 2022] DELETE /identity/v3/projects/8678c6cc8897418fb26532ec74186e15 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:23.337624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64d672-a093-447b-8e49-ccccb0056168 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.337941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64d672-a093-447b-8e49-ccccb0056168 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.341572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64d672-a093-447b-8e49-ccccb0056168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.358125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3d64d672-a093-447b-8e49-ccccb0056168 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:23.365326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3d64d672-a093-447b-8e49-ccccb0056168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:23.367740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 980/1937] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:23.377316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1346cc9-ad32-43d1-ab6f-d5c947a39510 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.377598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1346cc9-ad32-43d1-ab6f-d5c947a39510 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.377894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1346cc9-ad32-43d1-ab6f-d5c947a39510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.399541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a1346cc9-ad32-43d1-ab6f-d5c947a39510 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:23.400751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a1346cc9-ad32-43d1-ab6f-d5c947a39510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:23.415418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c733cf78a3874be595007784100c4f39', 'request_id': 'req-4d866dda-c5c9-449b-887b-dbc7ec112f3a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.475728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 981/1938] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:23.480630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1346cc9-ad32-43d1-ab6f-d5c947a39510 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.493954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1346cc9-ad32-43d1-ab6f-d5c947a39510 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.494871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c733cf78a3874be595007784100c4f39', 'request_id': 'req-4d866dda-c5c9-449b-887b-dbc7ec112f3a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.504658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.505352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.505641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.505946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.506614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:23.506922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.508525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.511835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c733cf78a3874be595007784100c4f39', 'request_id': 'req-4d866dda-c5c9-449b-887b-dbc7ec112f3a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.523458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 982/1939] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:23.527582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.532872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] The token cache is being invalidate because project c733cf78a3874be595007784100c4f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:23.533370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.534398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.536826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 958/1940] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:23 2022] DELETE /identity/v3/projects/c733cf78a3874be595007784100c4f39 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:23.538834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f32e45-e39c-4d2c-90e1-eac26d3dc009 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.541009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.546926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:23.547063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.547063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.547063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.547063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.547623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.547828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.565979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d866dda-c5c9-449b-887b-dbc7ec112f3a admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.565979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.566405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.566821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.567270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] PATH_INFO: `/v3/users/f38a5876e5ec46fb9758874b13d04976` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.568461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38a5876e5ec46fb9758874b13d04976)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.570458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.575795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:23.639122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 983/1941] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:23 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:23.644227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.645967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:23.714914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f3ba2-8be4-46c7-95d8-5e48d4ef68f3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.719886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae599b-7bda-4b21-9ec4-b40a2dd26304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.720589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae599b-7bda-4b21-9ec4-b40a2dd26304 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.720864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae599b-7bda-4b21-9ec4-b40a2dd26304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.721135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae599b-7bda-4b21-9ec4-b40a2dd26304 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.721774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae599b-7bda-4b21-9ec4-b40a2dd26304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.723697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae599b-7bda-4b21-9ec4-b40a2dd26304 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.825448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:23.837725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 984/1942] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:23 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:23.841600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae599b-7bda-4b21-9ec4-b40a2dd26304 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.852275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae599b-7bda-4b21-9ec4-b40a2dd26304 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.857253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489553-a6c0-4e9d-b0a1-4227709c8869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.858054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489553-a6c0-4e9d-b0a1-4227709c8869 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.858338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489553-a6c0-4e9d-b0a1-4227709c8869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.858615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489553-a6c0-4e9d-b0a1-4227709c8869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.859810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c489553-a6c0-4e9d-b0a1-4227709c8869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.861509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c489553-a6c0-4e9d-b0a1-4227709c8869 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.865080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38a5876e5ec46fb9758874b13d04976'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.874031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 985/1943] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:23.880700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489553-a6c0-4e9d-b0a1-4227709c8869 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.905079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489553-a6c0-4e9d-b0a1-4227709c8869 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.911380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53065ce-d034-4a93-9a28-db7f60cea532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.912295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53065ce-d034-4a93-9a28-db7f60cea532 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.912578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53065ce-d034-4a93-9a28-db7f60cea532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.912852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53065ce-d034-4a93-9a28-db7f60cea532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.913507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53065ce-d034-4a93-9a28-db7f60cea532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.915236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53065ce-d034-4a93-9a28-db7f60cea532 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.927516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 986/1944] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:23.931632 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53065ce-d034-4a93-9a28-db7f60cea532 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:23.939657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38a5876e5ec46fb9758874b13d04976'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.940345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38a5876e5ec46fb9758874b13d04976'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.944437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53065ce-d034-4a93-9a28-db7f60cea532 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:23.948104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:23.948856 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:23.949263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:23.949543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] PATH_INFO: `/v3/projects/9deebbe27272478a8ed5f1a02e2917b0/users/6aff7641471244a8b7e4b6d77c423cf0/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:23.952249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38a5876e5ec46fb9758874b13d04976'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.953961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3d0e5d10-ab23-4cb7-96ba-39897be63e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38a5876e5ec46fb9758874b13d04976'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:23.969557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9deebbe27272478a8ed5f1a02e2917b0, user_id=6aff7641471244a8b7e4b6d77c423cf0, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:23.972497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:23.978196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 959/1945] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:23 2022] DELETE /identity/v3/users/f38a5876e5ec46fb9758874b13d04976 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:24.055679 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 987/1946] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:23 2022] PUT /identity/v3/projects/9deebbe27272478a8ed5f1a02e2917b0/users/6aff7641471244a8b7e4b6d77c423cf0/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:24.067434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878108fc-899a-4ded-a7ec-f5b13577f948 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.067770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878108fc-899a-4ded-a7ec-f5b13577f948 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.068537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878108fc-899a-4ded-a7ec-f5b13577f948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.098994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-878108fc-899a-4ded-a7ec-f5b13577f948 None None] MFA Rules not processed for user `6aff7641471244a8b7e4b6d77c423cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:24.100224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-878108fc-899a-4ded-a7ec-f5b13577f948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:24.167174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 960/1947] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:24.172067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:24.174378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:24.269708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79dbd1-c16e-428b-a9eb-ce7a2d67ceec admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:24.275434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:24.276069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.276348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.276620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.277278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:24.279543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:24.422101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 988/1948] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:24 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:24.426519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878108fc-899a-4ded-a7ec-f5b13577f948 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:24.438741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878108fc-899a-4ded-a7ec-f5b13577f948 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:24.461035 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:24.464543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:24.512712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4662fe-8ee2-45a5-8145-0b3e8ccead52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:24.516010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4662fe-8ee2-45a5-8145-0b3e8ccead52 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.516010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4662fe-8ee2-45a5-8145-0b3e8ccead52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.516010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4662fe-8ee2-45a5-8145-0b3e8ccead52 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.516010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4662fe-8ee2-45a5-8145-0b3e8ccead52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:24.516560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4662fe-8ee2-45a5-8145-0b3e8ccead52 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:24.549740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2949322-488e-4434-bdbd-c34d31662fed admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:24.555863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:24.555863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.555863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.555863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] PATH_INFO: `/v3/projects/520b994e84594b81a9dc40600f54f244` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.557045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=520b994e84594b81a9dc40600f54f244)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:24.559147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:24.620752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 961/1949] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:24 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:24.625684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4662fe-8ee2-45a5-8145-0b3e8ccead52 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:24.631393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:24.641952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4662fe-8ee2-45a5-8145-0b3e8ccead52 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:24.642458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c3025-e572-49e0-8b7d-ceb58b108727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:24.643075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3025-e572-49e0-8b7d-ceb58b108727 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.643485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3025-e572-49e0-8b7d-ceb58b108727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.643848 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3025-e572-49e0-8b7d-ceb58b108727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.644514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c3025-e572-49e0-8b7d-ceb58b108727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:24.646130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c3025-e572-49e0-8b7d-ceb58b108727 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:24.657705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 962/1950] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:24.661679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c3025-e572-49e0-8b7d-ceb58b108727 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:24.672816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c3025-e572-49e0-8b7d-ceb58b108727 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:24.678094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361178f4-7a58-4817-aead-59bc227d32e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:24.678718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361178f4-7a58-4817-aead-59bc227d32e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.679000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361178f4-7a58-4817-aead-59bc227d32e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.679275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361178f4-7a58-4817-aead-59bc227d32e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.679962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361178f4-7a58-4817-aead-59bc227d32e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:24.681752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361178f4-7a58-4817-aead-59bc227d32e7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:24.694569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 963/1951] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:24.700683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361178f4-7a58-4817-aead-59bc227d32e7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:24.702073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:24.711936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361178f4-7a58-4817-aead-59bc227d32e7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:24.712257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520b994e84594b81a9dc40600f54f244', 'request_id': 'req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:24.779633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520b994e84594b81a9dc40600f54f244', 'request_id': 'req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:24.779959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6225c-63c7-4deb-9fb0-2b4b8bf49627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:24.780482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6225c-63c7-4deb-9fb0-2b4b8bf49627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.781155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6225c-63c7-4deb-9fb0-2b4b8bf49627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.781325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6225c-63c7-4deb-9fb0-2b4b8bf49627 admin admin] PATH_INFO: `/v3/projects/b854907e95da4cba8b0f5741e78037ed/users/b8068fa21355483f824d390bed76a1f8/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.791385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520b994e84594b81a9dc40600f54f244', 'request_id': 'req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:24.798852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6225c-63c7-4deb-9fb0-2b4b8bf49627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b854907e95da4cba8b0f5741e78037ed, user_id=b8068fa21355483f824d390bed76a1f8, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:24.801551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6225c-63c7-4deb-9fb0-2b4b8bf49627 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:24.804495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] The token cache is being invalidate because project 520b994e84594b81a9dc40600f54f244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:24.804892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:24.806263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a922e2e4-4bf6-45b9-a616-cf9c6e138dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:24.808339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 989/1952] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:24 2022] DELETE /identity/v3/projects/520b994e84594b81a9dc40600f54f244 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:24.815763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc360-49b7-4c8f-8e07-ac9f32d790ca None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.816057 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc360-49b7-4c8f-8e07-ac9f32d790ca None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.816329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc360-49b7-4c8f-8e07-ac9f32d790ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.837631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-401dc360-49b7-4c8f-8e07-ac9f32d790ca None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:24.839893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-401dc360-49b7-4c8f-8e07-ac9f32d790ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:24.852445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 990/1953] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:24.861537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685ce59-ab1d-49f3-b7b5-1bc1c6e1168b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.861823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685ce59-ab1d-49f3-b7b5-1bc1c6e1168b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.862123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685ce59-ab1d-49f3-b7b5-1bc1c6e1168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.885535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c685ce59-ab1d-49f3-b7b5-1bc1c6e1168b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:24.886802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c685ce59-ab1d-49f3-b7b5-1bc1c6e1168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:24.895093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 991/1954] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:24.902053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685ce59-ab1d-49f3-b7b5-1bc1c6e1168b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:24.915388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685ce59-ab1d-49f3-b7b5-1bc1c6e1168b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:24.920466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:24.920932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.921210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.922501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.923069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:24.923321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:24.924946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:24.938192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 992/1955] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:50:24.942536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:24.952403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 964/1956] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:24 2022] PUT /identity/v3/projects/b854907e95da4cba8b0f5741e78037ed/users/b8068fa21355483f824d390bed76a1f8/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:24.953934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96923fd9-9c75-4780-9a8e-40f17d67c769 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:24.963514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccd49-dcbe-4331-a1d5-7b50d0046747 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:24.965654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccd49-dcbe-4331-a1d5-7b50d0046747 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:24.965654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccd49-dcbe-4331-a1d5-7b50d0046747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:24.996730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d06ccd49-dcbe-4331-a1d5-7b50d0046747 None None] MFA Rules not processed for user `b8068fa21355483f824d390bed76a1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:24.998008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d06ccd49-dcbe-4331-a1d5-7b50d0046747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:25.023482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92bba0-ebdf-4bbd-951d-e93c116372b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.024102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92bba0-ebdf-4bbd-951d-e93c116372b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.024379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92bba0-ebdf-4bbd-951d-e93c116372b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.024650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92bba0-ebdf-4bbd-951d-e93c116372b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.025336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92bba0-ebdf-4bbd-951d-e93c116372b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.027283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92bba0-ebdf-4bbd-951d-e93c116372b1 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.073083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 965/1957] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:25.077933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ccd49-dcbe-4331-a1d5-7b50d0046747 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.083011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d06ccd49-dcbe-4331-a1d5-7b50d0046747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:25.104138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ccd49-dcbe-4331-a1d5-7b50d0046747 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.110757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a00013-7f63-40c5-9078-3f113edff838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.111386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a00013-7f63-40c5-9078-3f113edff838 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.111666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a00013-7f63-40c5-9078-3f113edff838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.111941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a00013-7f63-40c5-9078-3f113edff838 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.112579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a00013-7f63-40c5-9078-3f113edff838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.114670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a00013-7f63-40c5-9078-3f113edff838 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.136190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 993/1958] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:24 2022] POST /identity/v3/projects => generated 371 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:25.143418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92bba0-ebdf-4bbd-951d-e93c116372b1 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.216649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92bba0-ebdf-4bbd-951d-e93c116372b1 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.222392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f91c59-8b58-4a2f-8766-e490b9e8c10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.223027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91c59-8b58-4a2f-8766-e490b9e8c10b admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.223344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91c59-8b58-4a2f-8766-e490b9e8c10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.223736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91c59-8b58-4a2f-8766-e490b9e8c10b admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.224392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f91c59-8b58-4a2f-8766-e490b9e8c10b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.226365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f91c59-8b58-4a2f-8766-e490b9e8c10b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.240972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 966/1959] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:25 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:25.245404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a00013-7f63-40c5-9078-3f113edff838 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.285785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:25.314887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a00013-7f63-40c5-9078-3f113edff838 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.319625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e78a7-9295-4ac1-9eb8-29a51221a375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.320235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e78a7-9295-4ac1-9eb8-29a51221a375 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.320555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e78a7-9295-4ac1-9eb8-29a51221a375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.320833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e78a7-9295-4ac1-9eb8-29a51221a375 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.321464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e78a7-9295-4ac1-9eb8-29a51221a375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.323416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e78a7-9295-4ac1-9eb8-29a51221a375 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.339784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 994/1960] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:25 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:25.344716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f91c59-8b58-4a2f-8766-e490b9e8c10b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.354480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f91c59-8b58-4a2f-8766-e490b9e8c10b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.359318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f064f8a-b7c9-40f7-9bcf-99256c62c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.359913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064f8a-b7c9-40f7-9bcf-99256c62c1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.360197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064f8a-b7c9-40f7-9bcf-99256c62c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.360498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064f8a-b7c9-40f7-9bcf-99256c62c1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.361200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f064f8a-b7c9-40f7-9bcf-99256c62c1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.362826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f064f8a-b7c9-40f7-9bcf-99256c62c1b0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.373519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 995/1961] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:25.381547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f064f8a-b7c9-40f7-9bcf-99256c62c1b0 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.388163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:25.393225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f064f8a-b7c9-40f7-9bcf-99256c62c1b0 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.398349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c1008-0f31-4aba-a5d7-f4f2940d0bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.398977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c1008-0f31-4aba-a5d7-f4f2940d0bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.399256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c1008-0f31-4aba-a5d7-f4f2940d0bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.399531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c1008-0f31-4aba-a5d7-f4f2940d0bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.400192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c1008-0f31-4aba-a5d7-f4f2940d0bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.402295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c1008-0f31-4aba-a5d7-f4f2940d0bb2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.411266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 967/1962] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:25 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:25.416437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e78a7-9295-4ac1-9eb8-29a51221a375 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.420350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 996/1963] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:25.421244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c1008-0f31-4aba-a5d7-f4f2940d0bb2 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.428944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e78a7-9295-4ac1-9eb8-29a51221a375 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.433739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c1008-0f31-4aba-a5d7-f4f2940d0bb2 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.436474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a920e-f20d-4409-8407-8d893ec16859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.437533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a920e-f20d-4409-8407-8d893ec16859 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.438113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a920e-f20d-4409-8407-8d893ec16859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.438583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a920e-f20d-4409-8407-8d893ec16859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.439702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a920e-f20d-4409-8407-8d893ec16859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.441693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee44906-c57a-4e74-9681-f3ed6cd1cf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.442728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a920e-f20d-4409-8407-8d893ec16859 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.442825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44906-c57a-4e74-9681-f3ed6cd1cf49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.443207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44906-c57a-4e74-9681-f3ed6cd1cf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.443646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44906-c57a-4e74-9681-f3ed6cd1cf49 admin admin] PATH_INFO: `/v3/projects/523a84ad16e54e2aa69d18a695345e2f/users/f40dc2ffd2be451e83ff742cc903c581/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.457551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 968/1964] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:25.461829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a920e-f20d-4409-8407-8d893ec16859 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.464060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee44906-c57a-4e74-9681-f3ed6cd1cf49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523a84ad16e54e2aa69d18a695345e2f, user_id=f40dc2ffd2be451e83ff742cc903c581, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.466719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee44906-c57a-4e74-9681-f3ed6cd1cf49 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.473274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a920e-f20d-4409-8407-8d893ec16859 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.478244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51280ae5-749c-4898-a167-7f9c6cc48bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.478911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51280ae5-749c-4898-a167-7f9c6cc48bed admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.481595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51280ae5-749c-4898-a167-7f9c6cc48bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.481595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51280ae5-749c-4898-a167-7f9c6cc48bed admin admin] PATH_INFO: `/v3/projects/5ba517dfc15a4ca5bf3a98ed78108b5d/users/eb58f910203d4780a672104305751734/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.496540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51280ae5-749c-4898-a167-7f9c6cc48bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba517dfc15a4ca5bf3a98ed78108b5d, user_id=eb58f910203d4780a672104305751734, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.499460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51280ae5-749c-4898-a167-7f9c6cc48bed admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.615356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 997/1965] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:25 2022] PUT /identity/v3/projects/523a84ad16e54e2aa69d18a695345e2f/users/f40dc2ffd2be451e83ff742cc903c581/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:25.620844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0ede-4046-4b62-b4e0-6ac0323f6ca9 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.621134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0ede-4046-4b62-b4e0-6ac0323f6ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.621421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0ede-4046-4b62-b4e0-6ac0323f6ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.622199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 969/1966] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:25 2022] PUT /identity/v3/projects/5ba517dfc15a4ca5bf3a98ed78108b5d/users/eb58f910203d4780a672104305751734/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:25.626847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51280ae5-749c-4898-a167-7f9c6cc48bed admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.647157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-03cc0ede-4046-4b62-b4e0-6ac0323f6ca9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:25.648404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-03cc0ede-4046-4b62-b4e0-6ac0323f6ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:25.731158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51280ae5-749c-4898-a167-7f9c6cc48bed admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.744587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.744712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.745166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.745446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.746152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.747831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.758159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 998/1967] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:25.759885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 970/1968] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:25.767506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc0ede-4046-4b62-b4e0-6ac0323f6ca9 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.774590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.777523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:25.783933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc0ede-4046-4b62-b4e0-6ac0323f6ca9 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.789087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673d188-fede-4b53-b09e-b3b195bdeea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.790233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d188-fede-4b53-b09e-b3b195bdeea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.790597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d188-fede-4b53-b09e-b3b195bdeea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.791151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d188-fede-4b53-b09e-b3b195bdeea7 admin admin] PATH_INFO: `/v3/projects/5ba517dfc15a4ca5bf3a98ed78108b5d/users/eb58f910203d4780a672104305751734/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.793505 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673d188-fede-4b53-b09e-b3b195bdeea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba517dfc15a4ca5bf3a98ed78108b5d, user_id=eb58f910203d4780a672104305751734, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.797415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673d188-fede-4b53-b09e-b3b195bdeea7 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.811360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 999/1969] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:25 2022] PUT /identity/v3/projects/5ba517dfc15a4ca5bf3a98ed78108b5d/users/eb58f910203d4780a672104305751734/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:25.848862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff0bc4-fe4e-44d4-a5db-7feb0f26a0e4 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.887803 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:25.893616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be910eb1-161b-4afc-9317-185470348357 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.894122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be910eb1-161b-4afc-9317-185470348357 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.894596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be910eb1-161b-4afc-9317-185470348357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.920588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.921528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.921818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.922287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.945503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-be910eb1-161b-4afc-9317-185470348357 None None] MFA Rules not processed for user `eb58f910203d4780a672104305751734`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:25.951219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-be910eb1-161b-4afc-9317-185470348357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:26.027571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:26.029317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:26.043429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 971/1970] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:26.061584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1000/1971] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:26.076502 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:26.078620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:26.168206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3106-752b-4c06-88a5-0680d01a3d47 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:26.172664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bbeef-fa02-4fbc-927f-17eb52d32a76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:26.173156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bbeef-fa02-4fbc-927f-17eb52d32a76 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:26.173407 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bbeef-fa02-4fbc-927f-17eb52d32a76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:26.173688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bbeef-fa02-4fbc-927f-17eb52d32a76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:26.185971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bbeef-fa02-4fbc-927f-17eb52d32a76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:26.187611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bbeef-fa02-4fbc-927f-17eb52d32a76 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:26.209919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 972/1972] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:26.330231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be910eb1-161b-4afc-9317-185470348357 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:26.347025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be910eb1-161b-4afc-9317-185470348357 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:26.353316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:26.354147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:26.354525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:26.354861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:26.357709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:26.424922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:26.456011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:26.458550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:26.475374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1001/1973] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:27.588748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dd15-2ca9-4f7c-ae6a-b99cf8514d07 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:27.589281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dd15-2ca9-4f7c-ae6a-b99cf8514d07 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:27.589563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dd15-2ca9-4f7c-ae6a-b99cf8514d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:27.626065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d4e5dd15-2ca9-4f7c-ae6a-b99cf8514d07 None None] MFA Rules not processed for user `f40dc2ffd2be451e83ff742cc903c581`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:27.627904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d4e5dd15-2ca9-4f7c-ae6a-b99cf8514d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:27.726928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 973/1974] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:27.731893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:27.743316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c6c5d-0e4d-4780-b205-c362ba0d0ffa service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:27.748702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafc5bb-b74a-4d72-885b-dc4a87c58cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:27.749387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafc5bb-b74a-4d72-885b-dc4a87c58cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:27.749935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafc5bb-b74a-4d72-885b-dc4a87c58cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:27.750028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafc5bb-b74a-4d72-885b-dc4a87c58cd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:27.750718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafc5bb-b74a-4d72-885b-dc4a87c58cd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:27.752762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafc5bb-b74a-4d72-885b-dc4a87c58cd3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:27.885728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1002/1975] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:27 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:27.889495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5dd15-2ca9-4f7c-ae6a-b99cf8514d07 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:27.901549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5dd15-2ca9-4f7c-ae6a-b99cf8514d07 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:27.927088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:27.970994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4204ef-c2bf-43f9-925b-180a58df92d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:27.971665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4204ef-c2bf-43f9-925b-180a58df92d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:27.971972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4204ef-c2bf-43f9-925b-180a58df92d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:27.972136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4204ef-c2bf-43f9-925b-180a58df92d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:27.972885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4204ef-c2bf-43f9-925b-180a58df92d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:27.976306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4204ef-c2bf-43f9-925b-180a58df92d5 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.154244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 974/1976] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:27 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:28.158424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafc5bb-b74a-4d72-885b-dc4a87c58cd3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.170118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafc5bb-b74a-4d72-885b-dc4a87c58cd3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.175508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014daec4-ea64-486f-9df6-44b82a7bc4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.176143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014daec4-ea64-486f-9df6-44b82a7bc4ed admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.176421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014daec4-ea64-486f-9df6-44b82a7bc4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.176699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014daec4-ea64-486f-9df6-44b82a7bc4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.177364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014daec4-ea64-486f-9df6-44b82a7bc4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.179127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014daec4-ea64-486f-9df6-44b82a7bc4ed admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.191517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1003/1977] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:28.195895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4204ef-c2bf-43f9-925b-180a58df92d5 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.207074 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4204ef-c2bf-43f9-925b-180a58df92d5 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.212397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd16e22-6738-4a9e-89e1-d34bf75df31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.217437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd16e22-6738-4a9e-89e1-d34bf75df31e admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.217723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd16e22-6738-4a9e-89e1-d34bf75df31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.218118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd16e22-6738-4a9e-89e1-d34bf75df31e admin admin] PATH_INFO: `/v3/projects/9161e3d8c1214933a591db66c32d7c05/users/0cea4e5d6a3c4d0ab0d26873682243a2/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.235232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd16e22-6738-4a9e-89e1-d34bf75df31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9161e3d8c1214933a591db66c32d7c05, user_id=0cea4e5d6a3c4d0ab0d26873682243a2, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.238180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd16e22-6738-4a9e-89e1-d34bf75df31e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.351107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 975/1978] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:28 2022] PUT /identity/v3/projects/9161e3d8c1214933a591db66c32d7c05/users/0cea4e5d6a3c4d0ab0d26873682243a2/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:28.355182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014daec4-ea64-486f-9df6-44b82a7bc4ed admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.366219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014daec4-ea64-486f-9df6-44b82a7bc4ed admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.434849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d9adf-b118-420b-9d19-6db6dfee0353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.438210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d9adf-b118-420b-9d19-6db6dfee0353 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.438210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d9adf-b118-420b-9d19-6db6dfee0353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.438210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d9adf-b118-420b-9d19-6db6dfee0353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.438210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d9adf-b118-420b-9d19-6db6dfee0353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.440426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d9adf-b118-420b-9d19-6db6dfee0353 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.456540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1004/1979] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:28.458362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd16e22-6738-4a9e-89e1-d34bf75df31e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.492320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd16e22-6738-4a9e-89e1-d34bf75df31e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.508244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.509268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.509728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.510034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] PATH_INFO: `/v3/projects/9161e3d8c1214933a591db66c32d7c05/users/0cea4e5d6a3c4d0ab0d26873682243a2/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.518047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9161e3d8c1214933a591db66c32d7c05, user_id=0cea4e5d6a3c4d0ab0d26873682243a2, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.518047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.548339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 976/1980] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:28 2022] PUT /identity/v3/projects/9161e3d8c1214933a591db66c32d7c05/users/0cea4e5d6a3c4d0ab0d26873682243a2/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:30.194979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb239728-06ab-4f6e-9307-c52b98a4d978 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:30.195573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb239728-06ab-4f6e-9307-c52b98a4d978 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:30.196287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb239728-06ab-4f6e-9307-c52b98a4d978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:30.240073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-fb239728-06ab-4f6e-9307-c52b98a4d978 None None] MFA Rules not processed for user `0cea4e5d6a3c4d0ab0d26873682243a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:30.241598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-fb239728-06ab-4f6e-9307-c52b98a4d978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:30.322875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1005/1981] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:30.335285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:30.337332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:30.416558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e70869-1225-47a3-bde7-cc4b8d86a11b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:30.431796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95259a9e-e342-4a37-a07c-196f20ae9456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:30.431796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95259a9e-e342-4a37-a07c-196f20ae9456 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:30.431796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95259a9e-e342-4a37-a07c-196f20ae9456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:30.431796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95259a9e-e342-4a37-a07c-196f20ae9456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:30.524702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95259a9e-e342-4a37-a07c-196f20ae9456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:30.526228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95259a9e-e342-4a37-a07c-196f20ae9456 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:30.543340 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 977/1982] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:31.196030 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb239728-06ab-4f6e-9307-c52b98a4d978 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:31.228252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:31.260737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb239728-06ab-4f6e-9307-c52b98a4d978 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:31.266105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:31.266768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:31.267054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:31.267330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] PATH_INFO: `/v3/users/6aff7641471244a8b7e4b6d77c423cf0` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:31.270721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aff7641471244a8b7e4b6d77c423cf0)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:31.270721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:31.492901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aff7641471244a8b7e4b6d77c423cf0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:31.506244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aff7641471244a8b7e4b6d77c423cf0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:31.579778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aff7641471244a8b7e4b6d77c423cf0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:31.580409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aff7641471244a8b7e4b6d77c423cf0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:31.581060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aff7641471244a8b7e4b6d77c423cf0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:31.594712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1006/1983] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:31 2022] DELETE /identity/v3/users/6aff7641471244a8b7e4b6d77c423cf0 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:31.599478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268cb1d-68a0-4852-a688-fd18fb6113de None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:31.600017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268cb1d-68a0-4852-a688-fd18fb6113de None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:31.600017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268cb1d-68a0-4852-a688-fd18fb6113de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:31.620579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7268cb1d-68a0-4852-a688-fd18fb6113de None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:31.625222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7268cb1d-68a0-4852-a688-fd18fb6113de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:31.648691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:31.701786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 978/1984] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:31.710362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:31.711829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:31.787554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5811db-4ff2-4525-8395-d6058f825eed admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:31.791627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be28ad5-c9d6-479a-83dd-1921ff4c93d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:31.792299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be28ad5-c9d6-479a-83dd-1921ff4c93d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:31.792707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be28ad5-c9d6-479a-83dd-1921ff4c93d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:31.792996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be28ad5-c9d6-479a-83dd-1921ff4c93d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:31.804196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be28ad5-c9d6-479a-83dd-1921ff4c93d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:31.805981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be28ad5-c9d6-479a-83dd-1921ff4c93d5 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:31.822170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1007/1985] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:32.256344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268cb1d-68a0-4852-a688-fd18fb6113de None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:32.257788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7268cb1d-68a0-4852-a688-fd18fb6113de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:32.276770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268cb1d-68a0-4852-a688-fd18fb6113de None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:32.284531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:32.285331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:32.285538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:32.285815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] PATH_INFO: `/v3/projects/9deebbe27272478a8ed5f1a02e2917b0` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:32.287036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9deebbe27272478a8ed5f1a02e2917b0)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:32.289107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:32.395791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9deebbe27272478a8ed5f1a02e2917b0', 'request_id': 'req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.447912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9deebbe27272478a8ed5f1a02e2917b0', 'request_id': 'req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.455789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9deebbe27272478a8ed5f1a02e2917b0', 'request_id': 'req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.470987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] The token cache is being invalidate because project 9deebbe27272478a8ed5f1a02e2917b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:32.471280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.471958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.473552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 979/1986] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:32 2022] DELETE /identity/v3/projects/9deebbe27272478a8ed5f1a02e2917b0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:32.477620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be28ad5-c9d6-479a-83dd-1921ff4c93d5 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:32.488495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be28ad5-c9d6-479a-83dd-1921ff4c93d5 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:32.539461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:32.550622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:32.551266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:32.551549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:32.551831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] PATH_INFO: `/v3/users/b8068fa21355483f824d390bed76a1f8` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:32.553006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8068fa21355483f824d390bed76a1f8)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:32.555028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:32.753223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8068fa21355483f824d390bed76a1f8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.767095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8068fa21355483f824d390bed76a1f8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.842288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8068fa21355483f824d390bed76a1f8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.842959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8068fa21355483f824d390bed76a1f8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.843622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8068fa21355483f824d390bed76a1f8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.858727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1008/1987] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:32 2022] DELETE /identity/v3/users/b8068fa21355483f824d390bed76a1f8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:33.415997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:33.417473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:33.436661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3f92c-7d5d-4796-ad04-6753e30bd4d8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:33.514679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:33.515323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:33.515601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:33.518466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] PATH_INFO: `/v3/projects/b854907e95da4cba8b0f5741e78037ed` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:33.518466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b854907e95da4cba8b0f5741e78037ed)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:33.519129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:33.649953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b854907e95da4cba8b0f5741e78037ed', 'request_id': 'req-6af75236-7174-4369-a8c6-5f2473f501b9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.652571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:33.774109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b854907e95da4cba8b0f5741e78037ed', 'request_id': 'req-6af75236-7174-4369-a8c6-5f2473f501b9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.780943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b854907e95da4cba8b0f5741e78037ed', 'request_id': 'req-6af75236-7174-4369-a8c6-5f2473f501b9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.792760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] The token cache is being invalidate because project b854907e95da4cba8b0f5741e78037ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:33.793055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.793726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.795359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 980/1988] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:33 2022] DELETE /identity/v3/projects/b854907e95da4cba8b0f5741e78037ed => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:33.800869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:33.802393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:33.817007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98beb2d7-93c0-4189-8ad1-280af1a0aee2 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:33.878629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:33.879270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:33.879556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:33.879831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] PATH_INFO: `/v3/users/eb58f910203d4780a672104305751734` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:33.881003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb58f910203d4780a672104305751734)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:33.882982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:34.127973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb58f910203d4780a672104305751734'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.144946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb58f910203d4780a672104305751734'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.242531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb58f910203d4780a672104305751734'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.243051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb58f910203d4780a672104305751734'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.243659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb58f910203d4780a672104305751734'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.245431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:34.258888 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1009/1989] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:33 2022] DELETE /identity/v3/users/eb58f910203d4780a672104305751734 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:34.573410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:34.575328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:34.700397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af75236-7174-4369-a8c6-5f2473f501b9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:34.765366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9924853-ca89-4662-b13a-33ffd36f33b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:34.766056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9924853-ca89-4662-b13a-33ffd36f33b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:34.770446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9924853-ca89-4662-b13a-33ffd36f33b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:34.770446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9924853-ca89-4662-b13a-33ffd36f33b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:34.770446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b9924853-ca89-4662-b13a-33ffd36f33b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:34.804590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:34.807364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:34.869796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9924853-ca89-4662-b13a-33ffd36f33b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:34.872830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9924853-ca89-4662-b13a-33ffd36f33b4 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:34.894084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 981/1990] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:34.939492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26b479-cde3-484e-8861-d121499cd7f4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:34.944915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:34.945587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:34.945950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:34.946199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] PATH_INFO: `/v3/projects/5ba517dfc15a4ca5bf3a98ed78108b5d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:34.947516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba517dfc15a4ca5bf3a98ed78108b5d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:34.949723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:35.078293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba517dfc15a4ca5bf3a98ed78108b5d', 'request_id': 'req-b93f7eaa-706f-4bf3-9258-8138a4d95677'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:35.180370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba517dfc15a4ca5bf3a98ed78108b5d', 'request_id': 'req-b93f7eaa-706f-4bf3-9258-8138a4d95677'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:35.194684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba517dfc15a4ca5bf3a98ed78108b5d', 'request_id': 'req-b93f7eaa-706f-4bf3-9258-8138a4d95677'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:35.218369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] The token cache is being invalidate because project 5ba517dfc15a4ca5bf3a98ed78108b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:35.218883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:35.231193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b93f7eaa-706f-4bf3-9258-8138a4d95677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:35.233625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1010/1991] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:34 2022] DELETE /identity/v3/projects/5ba517dfc15a4ca5bf3a98ed78108b5d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:35.241212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a06587-1ce2-41ab-b7a3-a426af735c40 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:35.241573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a06587-1ce2-41ab-b7a3-a426af735c40 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:35.242176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a06587-1ce2-41ab-b7a3-a426af735c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:35.290590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-c5a06587-1ce2-41ab-b7a3-a426af735c40 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:35.290590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c5a06587-1ce2-41ab-b7a3-a426af735c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:35.355555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:35.379324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 982/1992] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:35.389557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9430b00-8099-412b-86fd-65076b796756 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:35.390093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9430b00-8099-412b-86fd-65076b796756 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:35.390595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9430b00-8099-412b-86fd-65076b796756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:35.433158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d9430b00-8099-412b-86fd-65076b796756 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:35.439828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d9430b00-8099-412b-86fd-65076b796756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:35.454963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1011/1993] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:35.467025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a06587-1ce2-41ab-b7a3-a426af735c40 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:35.504448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a06587-1ce2-41ab-b7a3-a426af735c40 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:35.526406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:35.526655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:35.526839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:35.535020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:35.535020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:35.535020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:35.535020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:35.557652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 983/1994] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:35.565178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9430b00-8099-412b-86fd-65076b796756 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:35.590333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9430b00-8099-412b-86fd-65076b796756 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:35.604205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862992b-8a2f-4d9c-93a9-29827d09de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:35.605226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862992b-8a2f-4d9c-93a9-29827d09de0b admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:35.605908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862992b-8a2f-4d9c-93a9-29827d09de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:35.606325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862992b-8a2f-4d9c-93a9-29827d09de0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:35.607009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862992b-8a2f-4d9c-93a9-29827d09de0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:35.613636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862992b-8a2f-4d9c-93a9-29827d09de0b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:35.761474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1012/1995] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:35 2022] POST /identity/v3/projects => generated 363 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:35.764940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:35.776584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924927c-ba4c-40af-a3ad-18becb5a16fb admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:35.856372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec31c60-3f05-4254-b9be-e29d2e7499bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:35.857008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31c60-3f05-4254-b9be-e29d2e7499bd admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:35.857403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31c60-3f05-4254-b9be-e29d2e7499bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:35.857678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31c60-3f05-4254-b9be-e29d2e7499bd admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:35.858387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec31c60-3f05-4254-b9be-e29d2e7499bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:35.861080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec31c60-3f05-4254-b9be-e29d2e7499bd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.017952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 984/1996] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:35 2022] POST /identity/v3/users => generated 282 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:36.022189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862992b-8a2f-4d9c-93a9-29827d09de0b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.034686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862992b-8a2f-4d9c-93a9-29827d09de0b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.039925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8330c-4224-4c77-bed0-5273b8c65576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.040595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8330c-4224-4c77-bed0-5273b8c65576 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.040856 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8330c-4224-4c77-bed0-5273b8c65576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.044765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8330c-4224-4c77-bed0-5273b8c65576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.044765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8330c-4224-4c77-bed0-5273b8c65576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.044765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8330c-4224-4c77-bed0-5273b8c65576 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.057684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1013/1997] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:36.061806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec31c60-3f05-4254-b9be-e29d2e7499bd admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.074525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec31c60-3f05-4254-b9be-e29d2e7499bd admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.080126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a86c0-40b3-4507-8a69-d02b174d7696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.080746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a86c0-40b3-4507-8a69-d02b174d7696 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.091153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a86c0-40b3-4507-8a69-d02b174d7696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.091153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a86c0-40b3-4507-8a69-d02b174d7696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.091153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a86c0-40b3-4507-8a69-d02b174d7696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.091153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a86c0-40b3-4507-8a69-d02b174d7696 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.096830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 985/1998] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:36.101391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8330c-4224-4c77-bed0-5273b8c65576 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.113574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8330c-4224-4c77-bed0-5273b8c65576 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.118477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.119917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.119917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.119917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] PATH_INFO: `/v3/projects/5ba927bd8fd242eeb9968bea40e7c19c/users/3a571a816c7141a4a3f49be082c47fca/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.141111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba927bd8fd242eeb9968bea40e7c19c, user_id=3a571a816c7141a4a3f49be082c47fca, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.148456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.249002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1014/1999] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:36 2022] PUT /identity/v3/projects/5ba927bd8fd242eeb9968bea40e7c19c/users/3a571a816c7141a4a3f49be082c47fca/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:36.262871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b12e8-3d4a-4396-a78c-bad5e726a725 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.263228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b12e8-3d4a-4396-a78c-bad5e726a725 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.263414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b12e8-3d4a-4396-a78c-bad5e726a725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.304008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ae2b12e8-3d4a-4396-a78c-bad5e726a725 None None] MFA Rules not processed for user `3a571a816c7141a4a3f49be082c47fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:36.305273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ae2b12e8-3d4a-4396-a78c-bad5e726a725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:36.400970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 986/2000] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:36.413939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.415803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:36.436583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:36.484299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d845f0d-6323-48c9-b6f6-6c9d9dd26502 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.487562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d204ec-f9c1-4184-8a46-eb9f7368c6ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.488191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d204ec-f9c1-4184-8a46-eb9f7368c6ca service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.488475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d204ec-f9c1-4184-8a46-eb9f7368c6ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.488752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d204ec-f9c1-4184-8a46-eb9f7368c6ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.497946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d204ec-f9c1-4184-8a46-eb9f7368c6ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.501095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d204ec-f9c1-4184-8a46-eb9f7368c6ca service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.511468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1015/2001] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:40.099768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b12e8-3d4a-4396-a78c-bad5e726a725 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:40.101236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ae2b12e8-3d4a-4396-a78c-bad5e726a725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:40.151226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:40.174250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b12e8-3d4a-4396-a78c-bad5e726a725 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:40.179397 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:40.180041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:40.180326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:40.180598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] PATH_INFO: `/v3/users/5e46a6823b28422fa176c73cd04cff4e` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:40.182130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e46a6823b28422fa176c73cd04cff4e)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:40.184301 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:40.431427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e46a6823b28422fa176c73cd04cff4e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:40.568598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e46a6823b28422fa176c73cd04cff4e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:40.569310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e46a6823b28422fa176c73cd04cff4e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:40.578243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e46a6823b28422fa176c73cd04cff4e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:40.578971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e46a6823b28422fa176c73cd04cff4e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:40.603555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 987/2002] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:40 2022] DELETE /identity/v3/users/5e46a6823b28422fa176c73cd04cff4e => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:40.607626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424f17-40f4-48c2-9333-d8b3092dff1b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:40.608482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424f17-40f4-48c2-9333-d8b3092dff1b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:40.608911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424f17-40f4-48c2-9333-d8b3092dff1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:40.633134 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b6424f17-40f4-48c2-9333-d8b3092dff1b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:40.636003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b6424f17-40f4-48c2-9333-d8b3092dff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:40.730283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1016/2003] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:40.740979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:40.742764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:40.845420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6c65e-88ba-4fe7-8212-6a0ea84781dc admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:40.849279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471e235-0ca6-4910-bc32-4d6da8a40e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:40.849949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e235-0ca6-4910-bc32-4d6da8a40e72 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:40.850261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e235-0ca6-4910-bc32-4d6da8a40e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:40.854443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e235-0ca6-4910-bc32-4d6da8a40e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:40.862131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471e235-0ca6-4910-bc32-4d6da8a40e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:40.864248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471e235-0ca6-4910-bc32-4d6da8a40e72 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:40.876830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 988/2004] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:41.539534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6424f17-40f4-48c2-9333-d8b3092dff1b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:41.540581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b6424f17-40f4-48c2-9333-d8b3092dff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:41.567657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6424f17-40f4-48c2-9333-d8b3092dff1b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:41.574332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:41.575007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:41.575307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:41.575581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] PATH_INFO: `/v3/projects/d614a23b1f164a67a833715689b2c1e4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:41.578209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d614a23b1f164a67a833715689b2c1e4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:41.580580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:41.771846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd614a23b1f164a67a833715689b2c1e4', 'request_id': 'req-547240f7-51de-4099-a4c1-8dd6440edffd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:41.866020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd614a23b1f164a67a833715689b2c1e4', 'request_id': 'req-547240f7-51de-4099-a4c1-8dd6440edffd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:41.875913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd614a23b1f164a67a833715689b2c1e4', 'request_id': 'req-547240f7-51de-4099-a4c1-8dd6440edffd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:41.889260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] The token cache is being invalidate because project d614a23b1f164a67a833715689b2c1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:41.889554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:41.890312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:41.892457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1017/2005] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:41 2022] DELETE /identity/v3/projects/d614a23b1f164a67a833715689b2c1e4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:41.897340 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471e235-0ca6-4910-bc32-4d6da8a40e72 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:41.908651 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471e235-0ca6-4910-bc32-4d6da8a40e72 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:41.979845 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:41.980495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:41.980776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:41.981195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] PATH_INFO: `/v3/users/f44f8fe20122489cb405f551465f8b2a` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:41.983239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44f8fe20122489cb405f551465f8b2a)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:41.986923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:41.992906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:42.254079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44f8fe20122489cb405f551465f8b2a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:42.333768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44f8fe20122489cb405f551465f8b2a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:42.334433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44f8fe20122489cb405f551465f8b2a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:42.342916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44f8fe20122489cb405f551465f8b2a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:42.344275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8531efaa-3958-436e-b54c-883410ae6c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44f8fe20122489cb405f551465f8b2a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:42.372903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 989/2006] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:41 2022] DELETE /identity/v3/users/f44f8fe20122489cb405f551465f8b2a => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:43.324839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:43.326430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:43.365410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547240f7-51de-4099-a4c1-8dd6440edffd admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:43.441801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:43.465928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:43.466842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:43.467277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:43.467704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] PATH_INFO: `/v3/projects/8f58c3168aca475d8555ea1548800b52` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:43.470494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f58c3168aca475d8555ea1548800b52)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:43.473372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:43.612011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f58c3168aca475d8555ea1548800b52', 'request_id': 'req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:43.733235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f58c3168aca475d8555ea1548800b52', 'request_id': 'req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:43.743104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f58c3168aca475d8555ea1548800b52', 'request_id': 'req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:43.763653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] The token cache is being invalidate because project 8f58c3168aca475d8555ea1548800b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:43.763960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:43.764769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b64994b-9fd4-4651-a9ad-fe6ed7faa9dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:43.768606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1018/2007] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:43 2022] DELETE /identity/v3/projects/8f58c3168aca475d8555ea1548800b52 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:43.773832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4c6d5-51a8-4a55-9aa7-606c6343c93a None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:43.774326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4c6d5-51a8-4a55-9aa7-606c6343c93a None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:43.774406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4c6d5-51a8-4a55-9aa7-606c6343c93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:43.796495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-cee4c6d5-51a8-4a55-9aa7-606c6343c93a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:43.798154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cee4c6d5-51a8-4a55-9aa7-606c6343c93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:43.877756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 990/2008] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:43.890223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6619-977e-42ff-8e03-d3f3eb34cb74 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:43.890600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6619-977e-42ff-8e03-d3f3eb34cb74 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:43.891500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6619-977e-42ff-8e03-d3f3eb34cb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:43.915832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-298f6619-977e-42ff-8e03-d3f3eb34cb74 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:43.917411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-298f6619-977e-42ff-8e03-d3f3eb34cb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:43.929671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1019/2009] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:43.932889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4c6d5-51a8-4a55-9aa7-606c6343c93a None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:43.954462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4c6d5-51a8-4a55-9aa7-606c6343c93a None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:43.978137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:43.979535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:43.980099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:43.980684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:43.981871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:43.982854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:43.994576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:44.073199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 991/2010] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:44.077872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f6619-977e-42ff-8e03-d3f3eb34cb74 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:44.090165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f6619-977e-42ff-8e03-d3f3eb34cb74 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:44.096041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:44.096748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.097037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.097315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.098004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:44.099964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:44.244251 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1020/2011] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:44 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:44.249001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:44.265525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a88e4-d836-4590-bd01-6405764a3d50 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:44.355987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16322d99-d480-4298-819e-10e7959b611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:44.356938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16322d99-d480-4298-819e-10e7959b611c admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.357326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16322d99-d480-4298-819e-10e7959b611c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.357486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16322d99-d480-4298-819e-10e7959b611c admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.358188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16322d99-d480-4298-819e-10e7959b611c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:44.360230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16322d99-d480-4298-819e-10e7959b611c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:44.367309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:44.369675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:44.385512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f5f65-ff00-481f-b492-4c5118566c6f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:44.392471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:44.393247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.393606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.393988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] PATH_INFO: `/v3/users/f5fcd46b400942a99dafdba6327ba49e` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.395569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5fcd46b400942a99dafdba6327ba49e)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:44.397601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:44.472998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:44.479776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 992/2012] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:44 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:44.481056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16322d99-d480-4298-819e-10e7959b611c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:44.493268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16322d99-d480-4298-819e-10e7959b611c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:44.501490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d4cca-c3d7-46cd-a1e3-3ac9923d8a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:44.502193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d4cca-c3d7-46cd-a1e3-3ac9923d8a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.502694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d4cca-c3d7-46cd-a1e3-3ac9923d8a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.503052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d4cca-c3d7-46cd-a1e3-3ac9923d8a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.503756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d4cca-c3d7-46cd-a1e3-3ac9923d8a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:44.505577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d4cca-c3d7-46cd-a1e3-3ac9923d8a8a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:44.523574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 993/2013] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:44.527383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d4cca-c3d7-46cd-a1e3-3ac9923d8a8a admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:44.538910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d4cca-c3d7-46cd-a1e3-3ac9923d8a8a admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:44.543773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0003954-0755-4226-8368-fcd9fd54a8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:44.544354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0003954-0755-4226-8368-fcd9fd54a8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.544636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0003954-0755-4226-8368-fcd9fd54a8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.544910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0003954-0755-4226-8368-fcd9fd54a8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.545558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0003954-0755-4226-8368-fcd9fd54a8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:44.550326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0003954-0755-4226-8368-fcd9fd54a8c7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:44.560549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 994/2014] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:44.564824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0003954-0755-4226-8368-fcd9fd54a8c7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:44.574724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0003954-0755-4226-8368-fcd9fd54a8c7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:44.580836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f0f77-71da-4f35-9f1a-052a5808f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:44.581522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f0f77-71da-4f35-9f1a-052a5808f5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.581797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f0f77-71da-4f35-9f1a-052a5808f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.582124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f0f77-71da-4f35-9f1a-052a5808f5a4 admin admin] PATH_INFO: `/v3/projects/fd42c656c6114451aad07c2c949cf9ee/users/b6a72055db574ade9f7dd164cbbdc9a1/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.598610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f0f77-71da-4f35-9f1a-052a5808f5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd42c656c6114451aad07c2c949cf9ee, user_id=b6a72055db574ade9f7dd164cbbdc9a1, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:44.601350 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f0f77-71da-4f35-9f1a-052a5808f5a4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:44.673501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5fcd46b400942a99dafdba6327ba49e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:44.689124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5fcd46b400942a99dafdba6327ba49e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:44.747721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 995/2015] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:44 2022] PUT /identity/v3/projects/fd42c656c6114451aad07c2c949cf9ee/users/b6a72055db574ade9f7dd164cbbdc9a1/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:44.753140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9821f8b-c8d9-4fa4-9bd8-49e0926493a0 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.753427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9821f8b-c8d9-4fa4-9bd8-49e0926493a0 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.753696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9821f8b-c8d9-4fa4-9bd8-49e0926493a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.786683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a9821f8b-c8d9-4fa4-9bd8-49e0926493a0 None None] MFA Rules not processed for user `b6a72055db574ade9f7dd164cbbdc9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:44.788084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a9821f8b-c8d9-4fa4-9bd8-49e0926493a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:44.858319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 996/2016] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:44.860334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5fcd46b400942a99dafdba6327ba49e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:44.860992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5fcd46b400942a99dafdba6327ba49e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:44.861656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5fcd46b400942a99dafdba6327ba49e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:44.863488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9821f8b-c8d9-4fa4-9bd8-49e0926493a0 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:44.883051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1021/2017] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:44 2022] DELETE /identity/v3/users/f5fcd46b400942a99dafdba6327ba49e => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:44.946530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9821f8b-c8d9-4fa4-9bd8-49e0926493a0 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.012197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.013101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.013101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.013368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.014073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.016057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.146444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 997/2018] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:44 2022] POST /identity/v3/projects => generated 371 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:45.151281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.151699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.153669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:45.153885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:45.187010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9127-c761-4de9-b884-a68876066d4a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.239571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f7ecc-58a9-49da-99bd-cab8b6cd0619 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.244863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa1c22-deab-46f3-bf3d-935754b8bf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.245567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa1c22-deab-46f3-bf3d-935754b8bf7c admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.245885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa1c22-deab-46f3-bf3d-935754b8bf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.246178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa1c22-deab-46f3-bf3d-935754b8bf7c admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.246880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa1c22-deab-46f3-bf3d-935754b8bf7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.248899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa1c22-deab-46f3-bf3d-935754b8bf7c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.277847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.278784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.279851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.279851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] PATH_INFO: `/v3/projects/b8a384a5f96e488fb3c4abd75fd876fb` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.281818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a384a5f96e488fb3c4abd75fd876fb)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.283966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.352041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 998/2019] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:45 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:45.356188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa1c22-deab-46f3-bf3d-935754b8bf7c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.367658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa1c22-deab-46f3-bf3d-935754b8bf7c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.372520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e3b8e-0087-43c7-9082-2afd6a3e258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.373127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3b8e-0087-43c7-9082-2afd6a3e258c admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.373404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3b8e-0087-43c7-9082-2afd6a3e258c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.373676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3b8e-0087-43c7-9082-2afd6a3e258c admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.374348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e3b8e-0087-43c7-9082-2afd6a3e258c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.376430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e3b8e-0087-43c7-9082-2afd6a3e258c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.387786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 999/2020] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:45.392282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e3b8e-0087-43c7-9082-2afd6a3e258c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.404839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a384a5f96e488fb3c4abd75fd876fb', 'request_id': 'req-e0aaeefb-47e0-4f2b-adb3-095183898e0d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:45.405895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e3b8e-0087-43c7-9082-2afd6a3e258c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.467938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.468584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.469445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.469716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] PATH_INFO: `/v3/projects/512d56dc79af4e6b97c19e4fb9848860/users/5847447919b84e8399ad146121e17a5c/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.484906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] RBAC: Authorizing `identity:create_grant(project_id=512d56dc79af4e6b97c19e4fb9848860, user_id=5847447919b84e8399ad146121e17a5c, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.487607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.510647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a384a5f96e488fb3c4abd75fd876fb', 'request_id': 'req-e0aaeefb-47e0-4f2b-adb3-095183898e0d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:45.519144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a384a5f96e488fb3c4abd75fd876fb', 'request_id': 'req-e0aaeefb-47e0-4f2b-adb3-095183898e0d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:45.533245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] The token cache is being invalidate because project b8a384a5f96e488fb3c4abd75fd876fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:45.533569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:45.534495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-e0aaeefb-47e0-4f2b-adb3-095183898e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:45.536489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1022/2021] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:45 2022] DELETE /identity/v3/projects/b8a384a5f96e488fb3c4abd75fd876fb => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:45.541813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9f4d5-ab0b-45d4-8208-8369847ac535 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.542135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9f4d5-ab0b-45d4-8208-8369847ac535 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.542417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9f4d5-ab0b-45d4-8208-8369847ac535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.562566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7bd9f4d5-ab0b-45d4-8208-8369847ac535 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:45.563999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7bd9f4d5-ab0b-45d4-8208-8369847ac535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:45.583950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1023/2022] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:45.583950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2b17-8011-4f36-bdcb-864325fdd378 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.583950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2b17-8011-4f36-bdcb-864325fdd378 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.583950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2b17-8011-4f36-bdcb-864325fdd378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.584409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1000/2023] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:45 2022] PUT /identity/v3/projects/512d56dc79af4e6b97c19e4fb9848860/users/5847447919b84e8399ad146121e17a5c/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:45.589224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.591282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:45.604464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ebdc2b17-8011-4f36-bdcb-864325fdd378 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:45.605674 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ebdc2b17-8011-4f36-bdcb-864325fdd378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:45.665393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626271a-99b1-4211-95cc-e7d88291355c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.670581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef295fed-7e82-4346-80bf-a33cc4f572ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.671199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef295fed-7e82-4346-80bf-a33cc4f572ee admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.671484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef295fed-7e82-4346-80bf-a33cc4f572ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.671820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef295fed-7e82-4346-80bf-a33cc4f572ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.673228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef295fed-7e82-4346-80bf-a33cc4f572ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.673376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1024/2024] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:45.674895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef295fed-7e82-4346-80bf-a33cc4f572ee admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.678058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc2b17-8011-4f36-bdcb-864325fdd378 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.685569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1001/2025] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:45.689477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:45.689952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef295fed-7e82-4346-80bf-a33cc4f572ee admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.692747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc2b17-8011-4f36-bdcb-864325fdd378 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.699047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.699670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.700169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.700279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef295fed-7e82-4346-80bf-a33cc4f572ee admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.700448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.701120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:45.701424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.703707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.705692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a3c86-1ae5-42f0-a68e-b89aedebdd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.706372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3c86-1ae5-42f0-a68e-b89aedebdd53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.706956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3c86-1ae5-42f0-a68e-b89aedebdd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.706956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3c86-1ae5-42f0-a68e-b89aedebdd53 admin admin] PATH_INFO: `/v3/projects/512d56dc79af4e6b97c19e4fb9848860/users/5847447919b84e8399ad146121e17a5c/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.708893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a3c86-1ae5-42f0-a68e-b89aedebdd53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512d56dc79af4e6b97c19e4fb9848860, user_id=5847447919b84e8399ad146121e17a5c, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.711606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a3c86-1ae5-42f0-a68e-b89aedebdd53 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.721062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1025/2026] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:50:45.722247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.724739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1002/2027] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:45 2022] PUT /identity/v3/projects/512d56dc79af4e6b97c19e4fb9848860/users/5847447919b84e8399ad146121e17a5c/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:45.732357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19600-d247-444d-aec8-f37f527e3ffd admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.741072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed053d-ca94-4dcc-9d9e-5bf35f73f4c0 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.741339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed053d-ca94-4dcc-9d9e-5bf35f73f4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.741684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed053d-ca94-4dcc-9d9e-5bf35f73f4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.773898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-eaed053d-ca94-4dcc-9d9e-5bf35f73f4c0 None None] MFA Rules not processed for user `5847447919b84e8399ad146121e17a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:45.775146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-eaed053d-ca94-4dcc-9d9e-5bf35f73f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:45.798352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.798895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.799235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.799515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.800222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:45.802716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:45.861738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1003/2028] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:45.924606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1026/2029] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:45 2022] POST /identity/v3/projects => generated 393 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:45.929075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed053d-ca94-4dcc-9d9e-5bf35f73f4c0 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.940705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed053d-ca94-4dcc-9d9e-5bf35f73f4c0 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.971712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.973426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.006145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de6c58-bcd1-47b0-9486-7f8e6db35290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.006760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de6c58-bcd1-47b0-9486-7f8e6db35290 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.007060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de6c58-bcd1-47b0-9486-7f8e6db35290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.007318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de6c58-bcd1-47b0-9486-7f8e6db35290 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.008037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de6c58-bcd1-47b0-9486-7f8e6db35290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.009970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de6c58-bcd1-47b0-9486-7f8e6db35290 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.041242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00a253-6a47-46b0-90f7-5526fa547cc0 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:46.045670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e099126-62f1-40ed-918c-f2cd4d98e472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.046359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e099126-62f1-40ed-918c-f2cd4d98e472 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.046640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e099126-62f1-40ed-918c-f2cd4d98e472 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.047579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e099126-62f1-40ed-918c-f2cd4d98e472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.108743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1004/2030] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:45 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:46.115675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de6c58-bcd1-47b0-9486-7f8e6db35290 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:46.126944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e099126-62f1-40ed-918c-f2cd4d98e472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.129037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e099126-62f1-40ed-918c-f2cd4d98e472 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.130427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de6c58-bcd1-47b0-9486-7f8e6db35290 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:46.135418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a612b8-deb7-4350-8081-eea62647f6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.136959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a612b8-deb7-4350-8081-eea62647f6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.136959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a612b8-deb7-4350-8081-eea62647f6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.136959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a612b8-deb7-4350-8081-eea62647f6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.137193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a612b8-deb7-4350-8081-eea62647f6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.138826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a612b8-deb7-4350-8081-eea62647f6c1 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.140435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1027/2031] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:46.149696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1005/2032] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:46.154498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e099126-62f1-40ed-918c-f2cd4d98e472 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:46.166036 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e099126-62f1-40ed-918c-f2cd4d98e472 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:46.170970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.171602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.172565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.173082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.174063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.175753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.189902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1028/2033] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:46.191608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a612b8-deb7-4350-8081-eea62647f6c1 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:46.205571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a612b8-deb7-4350-8081-eea62647f6c1 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:46.210672 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee713d-9274-4aab-b2c1-40f6b5ea2d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.213945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee713d-9274-4aab-b2c1-40f6b5ea2d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.214421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee713d-9274-4aab-b2c1-40f6b5ea2d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.215132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee713d-9274-4aab-b2c1-40f6b5ea2d5d admin admin] PATH_INFO: `/v3/projects/151e9d2b290047e8acd9468c721b9dd3/users/63ff0ea67ee64e14aafc5c51cad9684f/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.217692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:46.235263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee713d-9274-4aab-b2c1-40f6b5ea2d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=151e9d2b290047e8acd9468c721b9dd3, user_id=63ff0ea67ee64e14aafc5c51cad9684f, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.237913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee713d-9274-4aab-b2c1-40f6b5ea2d5d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.249245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:46.249245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.262596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb417d1-fbfe-4519-bddc-6ff4240ec606 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:46.273132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.276318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.276709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.277653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] PATH_INFO: `/v3/users/4adeaeb2f3ff4ad0aa5a853badde0c78` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.280001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4adeaeb2f3ff4ad0aa5a853badde0c78)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.284503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.369064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1006/2034] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:46 2022] PUT /identity/v3/projects/151e9d2b290047e8acd9468c721b9dd3/users/63ff0ea67ee64e14aafc5c51cad9684f/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:46.374141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa35441-6b14-4705-9e0d-54af102906c3 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.374511 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa35441-6b14-4705-9e0d-54af102906c3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.374875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa35441-6b14-4705-9e0d-54af102906c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.398566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2aa35441-6b14-4705-9e0d-54af102906c3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:46.399828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2aa35441-6b14-4705-9e0d-54af102906c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.468242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1007/2035] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:46.482026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa35441-6b14-4705-9e0d-54af102906c3 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:46.483472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2aa35441-6b14-4705-9e0d-54af102906c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.563677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa35441-6b14-4705-9e0d-54af102906c3 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:46.571023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad044a-7ba0-4df1-b660-d49381612328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.571268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad044a-7ba0-4df1-b660-d49381612328 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.571671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad044a-7ba0-4df1-b660-d49381612328 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.572259 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad044a-7ba0-4df1-b660-d49381612328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.575956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4adeaeb2f3ff4ad0aa5a853badde0c78'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:46.584843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad044a-7ba0-4df1-b660-d49381612328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.586345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad044a-7ba0-4df1-b660-d49381612328 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.599218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1008/2036] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:46.601099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4adeaeb2f3ff4ad0aa5a853badde0c78'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:46.705986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4adeaeb2f3ff4ad0aa5a853badde0c78'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:46.706330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4adeaeb2f3ff4ad0aa5a853badde0c78'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:46.707149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4adeaeb2f3ff4ad0aa5a853badde0c78'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:46.721174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1029/2037] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:46 2022] DELETE /identity/v3/users/4adeaeb2f3ff4ad0aa5a853badde0c78 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:46.726203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b79d-bcce-48da-810d-6d1bb513e035 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.726491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b79d-bcce-48da-810d-6d1bb513e035 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.726772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b79d-bcce-48da-810d-6d1bb513e035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.748934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a928b79d-bcce-48da-810d-6d1bb513e035 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:46.750937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a928b79d-bcce-48da-810d-6d1bb513e035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.819048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1009/2038] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:46.826740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:46.829088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.893402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02acb-f8e9-4319-a95e-ab6bff9007c5 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:46.897556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.898025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.898441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.898662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.908941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.910445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.920967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1030/2039] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:46.973241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:47.270600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b79d-bcce-48da-810d-6d1bb513e035 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:47.272390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a928b79d-bcce-48da-810d-6d1bb513e035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:47.281513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:47.286847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b79d-bcce-48da-810d-6d1bb513e035 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:47.291649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:47.293544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:47.293544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:47.293544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] PATH_INFO: `/v3/users/b6a72055db574ade9f7dd164cbbdc9a1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:47.294349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a72055db574ade9f7dd164cbbdc9a1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:47.296259 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:47.514082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a72055db574ade9f7dd164cbbdc9a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.654673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a72055db574ade9f7dd164cbbdc9a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.655655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a72055db574ade9f7dd164cbbdc9a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.669997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:47.671597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:47.672074 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a72055db574ade9f7dd164cbbdc9a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.673644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e86afb4d-4913-4ee0-92f8-bf18d0d51053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a72055db574ade9f7dd164cbbdc9a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.690762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dce982-9c47-4f3f-9311-407dc79a6bc4 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:47.703862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:47.704613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:47.704902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:47.705180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] PATH_INFO: `/v3/projects/8705d2ef80054da8b1257714d6af3907` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:47.711250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8705d2ef80054da8b1257714d6af3907)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:47.713227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:47.728870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1010/2040] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:47 2022] DELETE /identity/v3/users/b6a72055db574ade9f7dd164cbbdc9a1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:47.733572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b9aaa-9d86-4deb-83dd-5c25fd2e0717 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:47.733732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b9aaa-9d86-4deb-83dd-5c25fd2e0717 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:47.735130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b9aaa-9d86-4deb-83dd-5c25fd2e0717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:47.778063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f52b9aaa-9d86-4deb-83dd-5c25fd2e0717 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:47.780349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f52b9aaa-9d86-4deb-83dd-5c25fd2e0717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:47.864996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1011/2041] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:47.876559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b9aaa-9d86-4deb-83dd-5c25fd2e0717 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:47.878143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f52b9aaa-9d86-4deb-83dd-5c25fd2e0717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:47.885008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8705d2ef80054da8b1257714d6af3907', 'request_id': 'req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.946966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b9aaa-9d86-4deb-83dd-5c25fd2e0717 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.007975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.008381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8705d2ef80054da8b1257714d6af3907', 'request_id': 'req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.008694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.008866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.009138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.018669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8705d2ef80054da8b1257714d6af3907', 'request_id': 'req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.030011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] The token cache is being invalidate because project 8705d2ef80054da8b1257714d6af3907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:48.030304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.030968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1fafcf9e-87ec-486d-b121-70f71d6cf3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.033178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1031/2042] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:47 2022] DELETE /identity/v3/projects/8705d2ef80054da8b1257714d6af3907 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:48.038021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b197-c216-455c-ad78-ddcb000be478 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.038326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b197-c216-455c-ad78-ddcb000be478 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.038605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b197-c216-455c-ad78-ddcb000be478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.067820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-9974b197-c216-455c-ad78-ddcb000be478 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:48.069282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9974b197-c216-455c-ad78-ddcb000be478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:48.078590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1032/2043] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:48.088927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.089184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c1340-93a4-4e0c-b25a-a5266bc0b16b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.089620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c1340-93a4-4e0c-b25a-a5266bc0b16b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.090275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c1340-93a4-4e0c-b25a-a5266bc0b16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.090432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.103017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1012/2044] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:48.132436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-831c1340-93a4-4e0c-b25a-a5266bc0b16b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:48.136146 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-831c1340-93a4-4e0c-b25a-a5266bc0b16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:48.153923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1033/2045] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:48.155203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.156816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:48.170088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d62e9d-f5c5-4223-a263-e375a01796b6 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.175767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.176025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.176432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.177464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.177464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:48.177813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.179542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.193716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1013/2046] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:48.198809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c1340-93a4-4e0c-b25a-a5266bc0b16b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.210553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c1340-93a4-4e0c-b25a-a5266bc0b16b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.217257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177eb2-5c06-4c1b-91ab-684b60a8b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.218000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177eb2-5c06-4c1b-91ab-684b60a8b8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.218351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177eb2-5c06-4c1b-91ab-684b60a8b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.218691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177eb2-5c06-4c1b-91ab-684b60a8b8f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.219849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2177eb2-5c06-4c1b-91ab-684b60a8b8f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.221892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2177eb2-5c06-4c1b-91ab-684b60a8b8f4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.367123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1034/2047] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:48 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:48.371310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.383468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea86f6-d90b-4961-a545-8ddadabe22d8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.454486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:48.527179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e107a3-2cfb-4f36-b9f0-3ac8e5fbb1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.527874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e107a3-2cfb-4f36-b9f0-3ac8e5fbb1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.530638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e107a3-2cfb-4f36-b9f0-3ac8e5fbb1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.530638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e107a3-2cfb-4f36-b9f0-3ac8e5fbb1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.530638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e107a3-2cfb-4f36-b9f0-3ac8e5fbb1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.533079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e107a3-2cfb-4f36-b9f0-3ac8e5fbb1d3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.681835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1014/2048] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:48 2022] POST /identity/v3/users => generated 288 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:48.683028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177eb2-5c06-4c1b-91ab-684b60a8b8f4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.695010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177eb2-5c06-4c1b-91ab-684b60a8b8f4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.699799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.700415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.700692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.701007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.701747 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.704218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.714831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1035/2049] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:48.719195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e107a3-2cfb-4f36-b9f0-3ac8e5fbb1d3 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.732177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e107a3-2cfb-4f36-b9f0-3ac8e5fbb1d3 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.741017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.741948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f8850-2d18-4b52-8b71-a8810e500b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.742574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f8850-2d18-4b52-8b71-a8810e500b69 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.742854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f8850-2d18-4b52-8b71-a8810e500b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.743993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f8850-2d18-4b52-8b71-a8810e500b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.744680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f8850-2d18-4b52-8b71-a8810e500b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.745159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:48.746674 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f8850-2d18-4b52-8b71-a8810e500b69 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.764598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1015/2050] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:48.766422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f8850-2d18-4b52-8b71-a8810e500b69 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.780316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f8850-2d18-4b52-8b71-a8810e500b69 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.780707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9b374-b4e6-481a-b1e7-dbb674949dda admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.787322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680e0c7-3a5c-4947-81ab-13a87eb0b169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.788653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680e0c7-3a5c-4947-81ab-13a87eb0b169 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.788949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680e0c7-3a5c-4947-81ab-13a87eb0b169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.789302 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680e0c7-3a5c-4947-81ab-13a87eb0b169 admin admin] PATH_INFO: `/v3/projects/60c9a58027884a8e82f8c442d1a3474e/users/3247225f9b2244fe8b1b5cc2079e13cc/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.792611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.793212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.793491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.794186 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] PATH_INFO: `/v3/projects/fd42c656c6114451aad07c2c949cf9ee` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.795769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd42c656c6114451aad07c2c949cf9ee)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.798770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.815694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680e0c7-3a5c-4947-81ab-13a87eb0b169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c9a58027884a8e82f8c442d1a3474e, user_id=3247225f9b2244fe8b1b5cc2079e13cc, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.826620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680e0c7-3a5c-4947-81ab-13a87eb0b169 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.882458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:48.924825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1016/2051] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:48 2022] PUT /identity/v3/projects/60c9a58027884a8e82f8c442d1a3474e/users/3247225f9b2244fe8b1b5cc2079e13cc/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:48.935365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59c86-470b-4601-80a3-2d4e99ce44f7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.935585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59c86-470b-4601-80a3-2d4e99ce44f7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.935861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59c86-470b-4601-80a3-2d4e99ce44f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.970435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd42c656c6114451aad07c2c949cf9ee', 'request_id': 'req-5162e240-5695-47d3-b459-e7605abe99cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.980545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-8eb59c86-470b-4601-80a3-2d4e99ce44f7 None None] MFA Rules not processed for user `3247225f9b2244fe8b1b5cc2079e13cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:48.981796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8eb59c86-470b-4601-80a3-2d4e99ce44f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.054546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd42c656c6114451aad07c2c949cf9ee', 'request_id': 'req-5162e240-5695-47d3-b459-e7605abe99cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.066462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1017/2052] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:49.067554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59c86-470b-4601-80a3-2d4e99ce44f7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.068214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd42c656c6114451aad07c2c949cf9ee', 'request_id': 'req-5162e240-5695-47d3-b459-e7605abe99cb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.090865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] The token cache is being invalidate because project fd42c656c6114451aad07c2c949cf9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:49.091707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.093056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.096668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1036/2053] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:48 2022] DELETE /identity/v3/projects/fd42c656c6114451aad07c2c949cf9ee => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:49.100588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.102471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.147246 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59c86-470b-4601-80a3-2d4e99ce44f7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.153039 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.154237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.154520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.154797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.155474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.158178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.202275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162e240-5695-47d3-b459-e7605abe99cb admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.215271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.216123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.216421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.216703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] PATH_INFO: `/v3/users/5847447919b84e8399ad146121e17a5c` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.219490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5847447919b84e8399ad146121e17a5c)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.221635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.294678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1018/2054] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/projects => generated 375 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:49.300266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.303134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.400644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7176d-57a9-4276-923f-a5c0c5d7f8b4 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.409396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccd76f-139e-4129-a38d-cb139b7e6110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.410178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccd76f-139e-4129-a38d-cb139b7e6110 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.413643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccd76f-139e-4129-a38d-cb139b7e6110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.413643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccd76f-139e-4129-a38d-cb139b7e6110 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.413643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccd76f-139e-4129-a38d-cb139b7e6110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.421388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccd76f-139e-4129-a38d-cb139b7e6110 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.481016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5847447919b84e8399ad146121e17a5c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.498437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5847447919b84e8399ad146121e17a5c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.550566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1019/2055] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/users => generated 294 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:49.555383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccd76f-139e-4129-a38d-cb139b7e6110 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.569118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccd76f-139e-4129-a38d-cb139b7e6110 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.576853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.579210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.579799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.580208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.581042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.582988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.587760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5847447919b84e8399ad146121e17a5c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.588415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5847447919b84e8399ad146121e17a5c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.590740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5847447919b84e8399ad146121e17a5c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.599381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1020/2056] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:49.603607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.608806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.639208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff67ca1-006d-4220-b2c3-d8f1ac112471 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.647627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1037/2057] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:49 2022] DELETE /identity/v3/users/5847447919b84e8399ad146121e17a5c => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:49.726840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d52ed8-ebfa-4778-99ca-8f9185f6a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.730641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d52ed8-ebfa-4778-99ca-8f9185f6a5a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.730970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d52ed8-ebfa-4778-99ca-8f9185f6a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.731248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d52ed8-ebfa-4778-99ca-8f9185f6a5a2 admin admin] PATH_INFO: `/v3/projects/0ffdd0764f8b4726ab1e02c86aecbaf3/users/40b559fef39d4a3a92fe222f1e138876/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.752748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d52ed8-ebfa-4778-99ca-8f9185f6a5a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ffdd0764f8b4726ab1e02c86aecbaf3, user_id=40b559fef39d4a3a92fe222f1e138876, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.756039 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d52ed8-ebfa-4778-99ca-8f9185f6a5a2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.934713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1021/2058] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:49 2022] PUT /identity/v3/projects/0ffdd0764f8b4726ab1e02c86aecbaf3/users/40b559fef39d4a3a92fe222f1e138876/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:49.939508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.955234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1a9e9-d989-4f14-87e9-7cec868759bc admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.968232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:50.059956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ed543-c800-4554-bcde-401975e2c555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.061015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ed543-c800-4554-bcde-401975e2c555 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.062909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ed543-c800-4554-bcde-401975e2c555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.063738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ed543-c800-4554-bcde-401975e2c555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.064894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ed543-c800-4554-bcde-401975e2c555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.068218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ed543-c800-4554-bcde-401975e2c555 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.086094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1038/2059] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:50.091808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d52ed8-ebfa-4778-99ca-8f9185f6a5a2 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.103643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d52ed8-ebfa-4778-99ca-8f9185f6a5a2 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.108965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.110496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.110786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.111094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] PATH_INFO: `/v3/projects/0ffdd0764f8b4726ab1e02c86aecbaf3/users/40b559fef39d4a3a92fe222f1e138876/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.113270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ffdd0764f8b4726ab1e02c86aecbaf3, user_id=40b559fef39d4a3a92fe222f1e138876, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.116036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.134911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1022/2060] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:50 2022] PUT /identity/v3/projects/0ffdd0764f8b4726ab1e02c86aecbaf3/users/40b559fef39d4a3a92fe222f1e138876/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:50.146698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d4c68-b938-420a-bc55-d37e65dfb9ae None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.150386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d4c68-b938-420a-bc55-d37e65dfb9ae None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.150755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d4c68-b938-420a-bc55-d37e65dfb9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.202695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-420d4c68-b938-420a-bc55-d37e65dfb9ae None None] MFA Rules not processed for user `40b559fef39d4a3a92fe222f1e138876`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:50.204488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-420d4c68-b938-420a-bc55-d37e65dfb9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:50.301254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1039/2061] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:50.314190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.315997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:50.392494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca037-d145-46b7-98db-1203985f9f6d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.397744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64a67a-d368-4f92-9cd4-04339d00177b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.398588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a67a-d368-4f92-9cd4-04339d00177b service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.398875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a67a-d368-4f92-9cd4-04339d00177b service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.399176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a67a-d368-4f92-9cd4-04339d00177b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.401196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8b64a67a-d368-4f92-9cd4-04339d00177b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:50.512116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64a67a-d368-4f92-9cd4-04339d00177b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.515757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64a67a-d368-4f92-9cd4-04339d00177b service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.536974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1023/2062] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:50.541197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d4c68-b938-420a-bc55-d37e65dfb9ae None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.546139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-420d4c68-b938-420a-bc55-d37e65dfb9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:50.754080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d4c68-b938-420a-bc55-d37e65dfb9ae None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.766215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.766777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.767060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.767338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] PATH_INFO: `/v3/projects/512d56dc79af4e6b97c19e4fb9848860` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.768532 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=512d56dc79af4e6b97c19e4fb9848860)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.770498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.814474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:50.962381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512d56dc79af4e6b97c19e4fb9848860', 'request_id': 'req-999a1016-d899-4692-95c2-36bfcf90f1aa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:51.065160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512d56dc79af4e6b97c19e4fb9848860', 'request_id': 'req-999a1016-d899-4692-95c2-36bfcf90f1aa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:51.074300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512d56dc79af4e6b97c19e4fb9848860', 'request_id': 'req-999a1016-d899-4692-95c2-36bfcf90f1aa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:51.087212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] The token cache is being invalidate because project 512d56dc79af4e6b97c19e4fb9848860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:51.087501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:51.088311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-999a1016-d899-4692-95c2-36bfcf90f1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:51.089825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1040/2063] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:50 2022] DELETE /identity/v3/projects/512d56dc79af4e6b97c19e4fb9848860 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:51.096647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a4557-4750-4e7a-a0d4-2a846e5202b9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:51.096951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a4557-4750-4e7a-a0d4-2a846e5202b9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:51.097252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a4557-4750-4e7a-a0d4-2a846e5202b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:51.120910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-062a4557-4750-4e7a-a0d4-2a846e5202b9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:51.123146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-062a4557-4750-4e7a-a0d4-2a846e5202b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:51.230272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1024/2064] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:51.241918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c63278-6b39-40c0-a67f-bd348e7e657f None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:51.242753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c63278-6b39-40c0-a67f-bd348e7e657f None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:51.243066 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c63278-6b39-40c0-a67f-bd348e7e657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:51.280205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a5c63278-6b39-40c0-a67f-bd348e7e657f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:51.281763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a5c63278-6b39-40c0-a67f-bd348e7e657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:51.288081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:51.290844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1041/2065] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:51.298469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a4557-4750-4e7a-a0d4-2a846e5202b9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:51.305992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a4557-4750-4e7a-a0d4-2a846e5202b9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:51.310754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:51.311370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:51.311648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:51.311915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:51.313099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:51.313400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:51.315082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:51.329511 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1025/2066] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:50:51.336821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c63278-6b39-40c0-a67f-bd348e7e657f None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:51.348623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c63278-6b39-40c0-a67f-bd348e7e657f None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:51.357638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1c504-c336-45ce-8e96-ff36246fbf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:51.357638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1c504-c336-45ce-8e96-ff36246fbf8a admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:51.357638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1c504-c336-45ce-8e96-ff36246fbf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:51.358288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1c504-c336-45ce-8e96-ff36246fbf8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:51.359323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1c504-c336-45ce-8e96-ff36246fbf8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:51.363100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1c504-c336-45ce-8e96-ff36246fbf8a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:51.574218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:51.574923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1042/2067] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:51 2022] POST /identity/v3/projects => generated 355 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:51.604825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ccb7b-954e-4f22-a68a-ad3d219be619 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:51.737679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7163f-8551-42b5-bae2-05e6f48caf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:51.739864 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7163f-8551-42b5-bae2-05e6f48caf30 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:51.740419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7163f-8551-42b5-bae2-05e6f48caf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:51.740705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7163f-8551-42b5-bae2-05e6f48caf30 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:51.741374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7163f-8551-42b5-bae2-05e6f48caf30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:51.745010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7163f-8551-42b5-bae2-05e6f48caf30 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:51.845478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1026/2068] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:51 2022] POST /identity/v3/users => generated 278 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:51.849478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1c504-c336-45ce-8e96-ff36246fbf8a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:51.891587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1c504-c336-45ce-8e96-ff36246fbf8a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:51.929075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b3f50-f92c-415f-9e56-e47bf6670af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:51.929894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b3f50-f92c-415f-9e56-e47bf6670af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:51.930102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b3f50-f92c-415f-9e56-e47bf6670af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:51.930379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b3f50-f92c-415f-9e56-e47bf6670af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:51.931062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b3f50-f92c-415f-9e56-e47bf6670af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:51.933191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b3f50-f92c-415f-9e56-e47bf6670af4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:51.951441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1043/2069] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:51.958749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7163f-8551-42b5-bae2-05e6f48caf30 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:51.977470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7163f-8551-42b5-bae2-05e6f48caf30 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:51.982973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea7900-e287-45ac-96be-4b2f3945b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:51.983600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7900-e287-45ac-96be-4b2f3945b071 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:51.983936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7900-e287-45ac-96be-4b2f3945b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:51.984252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7900-e287-45ac-96be-4b2f3945b071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:51.984970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea7900-e287-45ac-96be-4b2f3945b071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:51.987796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea7900-e287-45ac-96be-4b2f3945b071 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:52.016070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b3f50-f92c-415f-9e56-e47bf6670af4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:52.023295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1027/2070] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:52.027439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b3f50-f92c-415f-9e56-e47bf6670af4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:52.035205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da1197-f2ba-4a48-a1cc-ef0f0d587a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:52.036256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da1197-f2ba-4a48-a1cc-ef0f0d587a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:52.036744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da1197-f2ba-4a48-a1cc-ef0f0d587a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:52.037217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da1197-f2ba-4a48-a1cc-ef0f0d587a18 admin admin] PATH_INFO: `/v3/projects/ebd39a61abf04a2f9c344a81e83eabb2/users/1c65965f8254439d94acc81461d443d6/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:52.060361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da1197-f2ba-4a48-a1cc-ef0f0d587a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd39a61abf04a2f9c344a81e83eabb2, user_id=1c65965f8254439d94acc81461d443d6, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:52.064119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da1197-f2ba-4a48-a1cc-ef0f0d587a18 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:52.146596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1044/2071] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:52 2022] PUT /identity/v3/projects/ebd39a61abf04a2f9c344a81e83eabb2/users/1c65965f8254439d94acc81461d443d6/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:52.159531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92f0c-fdf6-434a-9d0d-b51de98b18bd None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:52.159832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92f0c-fdf6-434a-9d0d-b51de98b18bd None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:52.160274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92f0c-fdf6-434a-9d0d-b51de98b18bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:52.240266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-c4f92f0c-fdf6-434a-9d0d-b51de98b18bd None None] MFA Rules not processed for user `1c65965f8254439d94acc81461d443d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:52.241833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c4f92f0c-fdf6-434a-9d0d-b51de98b18bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:52.362882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1028/2072] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:52.367999 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da1197-f2ba-4a48-a1cc-ef0f0d587a18 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:52.441116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da1197-f2ba-4a48-a1cc-ef0f0d587a18 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:52.446113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4662b-56b6-4351-8cca-2f025349564e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:52.447029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4662b-56b6-4351-8cca-2f025349564e admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:52.447325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4662b-56b6-4351-8cca-2f025349564e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:52.447602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4662b-56b6-4351-8cca-2f025349564e admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:52.448252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4662b-56b6-4351-8cca-2f025349564e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:52.450230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4662b-56b6-4351-8cca-2f025349564e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:52.577426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1045/2073] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:52 2022] POST /identity/v3/projects => generated 355 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:52.583658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f92f0c-fdf6-434a-9d0d-b51de98b18bd None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:52.600807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f92f0c-fdf6-434a-9d0d-b51de98b18bd None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:52.754051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4e1e4-c445-4639-8746-8e90fd0480ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:52.754989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e1e4-c445-4639-8746-8e90fd0480ac admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:52.755388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e1e4-c445-4639-8746-8e90fd0480ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:52.755715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e1e4-c445-4639-8746-8e90fd0480ac admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:52.756441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4e1e4-c445-4639-8746-8e90fd0480ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:52.758863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4e1e4-c445-4639-8746-8e90fd0480ac admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:52.931421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1029/2074] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:52 2022] POST /identity/v3/users => generated 284 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:52.932639 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4662b-56b6-4351-8cca-2f025349564e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:52.954846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4662b-56b6-4351-8cca-2f025349564e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:52.963069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374affe-a4bd-4b6a-bc7d-e9a10663030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:52.964733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374affe-a4bd-4b6a-bc7d-e9a10663030b admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:52.965022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374affe-a4bd-4b6a-bc7d-e9a10663030b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:52.965299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374affe-a4bd-4b6a-bc7d-e9a10663030b admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:52.966026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374affe-a4bd-4b6a-bc7d-e9a10663030b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:52.967844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374affe-a4bd-4b6a-bc7d-e9a10663030b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:52.979955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1046/2075] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:52.985455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4e1e4-c445-4639-8746-8e90fd0480ac admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:53.004023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4e1e4-c445-4639-8746-8e90fd0480ac admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:53.011891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098e853-9f5d-42fb-a38e-bb66302cc706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:53.012826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098e853-9f5d-42fb-a38e-bb66302cc706 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:53.013325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098e853-9f5d-42fb-a38e-bb66302cc706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:53.013797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098e853-9f5d-42fb-a38e-bb66302cc706 admin admin] PATH_INFO: `/v3/projects/c05acfb001f0476d906f99461b9121e3/users/601c45ad63ec4938b1f79deade2e54e7/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:53.037234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098e853-9f5d-42fb-a38e-bb66302cc706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05acfb001f0476d906f99461b9121e3, user_id=601c45ad63ec4938b1f79deade2e54e7, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:53.044227 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098e853-9f5d-42fb-a38e-bb66302cc706 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:53.181453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1030/2076] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:52 2022] PUT /identity/v3/projects/c05acfb001f0476d906f99461b9121e3/users/601c45ad63ec4938b1f79deade2e54e7/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:53.185438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374affe-a4bd-4b6a-bc7d-e9a10663030b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:53.196407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374affe-a4bd-4b6a-bc7d-e9a10663030b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:53.205304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:53.296517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13d83d-7d5f-4ac2-b96c-2e8be3816ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:53.297383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13d83d-7d5f-4ac2-b96c-2e8be3816ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:53.297739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13d83d-7d5f-4ac2-b96c-2e8be3816ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:53.298372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13d83d-7d5f-4ac2-b96c-2e8be3816ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:53.299404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13d83d-7d5f-4ac2-b96c-2e8be3816ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:53.301977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13d83d-7d5f-4ac2-b96c-2e8be3816ba9 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:53.319536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1047/2077] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:53.326016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098e853-9f5d-42fb-a38e-bb66302cc706 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:53.350462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098e853-9f5d-42fb-a38e-bb66302cc706 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:53.361160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:53.362410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:53.362940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:53.363415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] PATH_INFO: `/v3/projects/c05acfb001f0476d906f99461b9121e3/users/601c45ad63ec4938b1f79deade2e54e7/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:53.367307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05acfb001f0476d906f99461b9121e3, user_id=601c45ad63ec4938b1f79deade2e54e7, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:53.376859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:53.399659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1031/2078] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:53 2022] PUT /identity/v3/projects/c05acfb001f0476d906f99461b9121e3/users/601c45ad63ec4938b1f79deade2e54e7/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:53.414658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea988b-cabf-4895-9438-749c667f8b8b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:53.415132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea988b-cabf-4895-9438-749c667f8b8b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:53.415861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea988b-cabf-4895-9438-749c667f8b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:53.462821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1bea988b-cabf-4895-9438-749c667f8b8b None None] MFA Rules not processed for user `601c45ad63ec4938b1f79deade2e54e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:53.464161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1bea988b-cabf-4895-9438-749c667f8b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:53.596680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1048/2079] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:53.620920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:53.624396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:53.745587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c62a3-53aa-4f20-be5e-2fb4d1a6dcfb admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:53.751108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed4ba1-af21-497b-8f63-3de97091985b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:53.751546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4ba1-af21-497b-8f63-3de97091985b service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:53.751817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4ba1-af21-497b-8f63-3de97091985b service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:53.752117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4ba1-af21-497b-8f63-3de97091985b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:53.767303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4ba1-af21-497b-8f63-3de97091985b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:53.774398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4ba1-af21-497b-8f63-3de97091985b service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:53.809690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1032/2080] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:53.934386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:54.022584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5194-7ad5-4dcf-af6a-a5c319c1c0d2 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:54.023032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5194-7ad5-4dcf-af6a-a5c319c1c0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:54.023461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5194-7ad5-4dcf-af6a-a5c319c1c0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:54.063521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-0fdc5194-7ad5-4dcf-af6a-a5c319c1c0d2 None None] MFA Rules not processed for user `63ff0ea67ee64e14aafc5c51cad9684f`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:54.064806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0fdc5194-7ad5-4dcf-af6a-a5c319c1c0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:54.135592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1049/2081] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:54.147587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04823-85b9-4411-9693-ebf7b890da95 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:54.150189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04823-85b9-4411-9693-ebf7b890da95 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:54.150189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04823-85b9-4411-9693-ebf7b890da95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:54.178718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-eea04823-85b9-4411-9693-ebf7b890da95 None None] MFA Rules not processed for user `63ff0ea67ee64e14aafc5c51cad9684f`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:54.183637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-eea04823-85b9-4411-9693-ebf7b890da95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:54.186130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5194-7ad5-4dcf-af6a-a5c319c1c0d2 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:54.187587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0fdc5194-7ad5-4dcf-af6a-a5c319c1c0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:54.192252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1033/2082] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:54.200238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5194-7ad5-4dcf-af6a-a5c319c1c0d2 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:54.204271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:54.204705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:54.205002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:54.205308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:54.207991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04823-85b9-4411-9693-ebf7b890da95 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:54.209431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-eea04823-85b9-4411-9693-ebf7b890da95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:54.295315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:54.298147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:54.310659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1050/2083] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:54.318207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04823-85b9-4411-9693-ebf7b890da95 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:54.334845 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe88b14-db49-4a2d-8f69-41cd49eeb027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:54.336383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe88b14-db49-4a2d-8f69-41cd49eeb027 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:54.336724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe88b14-db49-4a2d-8f69-41cd49eeb027 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:54.337192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe88b14-db49-4a2d-8f69-41cd49eeb027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:54.361278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe88b14-db49-4a2d-8f69-41cd49eeb027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:54.363679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe88b14-db49-4a2d-8f69-41cd49eeb027 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:54.378702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:54.380611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:54.388426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1034/2084] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:54.540575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4f801-d57c-4fb5-99b9-5684fc6dd910 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:54.547597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:54.548383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:54.548739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:54.549075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] PATH_INFO: `/v3/users/3247225f9b2244fe8b1b5cc2079e13cc` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:54.550780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3247225f9b2244fe8b1b5cc2079e13cc)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:54.552965 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:54.852557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3247225f9b2244fe8b1b5cc2079e13cc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:54.868936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3247225f9b2244fe8b1b5cc2079e13cc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:54.989676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3247225f9b2244fe8b1b5cc2079e13cc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:54.991018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3247225f9b2244fe8b1b5cc2079e13cc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:54.992158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-986902af-2ccf-4523-abcf-45aff95dd076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3247225f9b2244fe8b1b5cc2079e13cc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:55.020919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1051/2085] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:54 2022] DELETE /identity/v3/users/3247225f9b2244fe8b1b5cc2079e13cc => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:55.021402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4838af7-0c3b-4a01-8e89-49ff03ca8679 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:55.021807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4838af7-0c3b-4a01-8e89-49ff03ca8679 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:55.022356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4838af7-0c3b-4a01-8e89-49ff03ca8679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:55.050659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-c4838af7-0c3b-4a01-8e89-49ff03ca8679 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:55.051979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c4838af7-0c3b-4a01-8e89-49ff03ca8679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:55.123590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b743-dc60-4050-84a6-fff990fc0d3b None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:55.123877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b743-dc60-4050-84a6-fff990fc0d3b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:55.124344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b743-dc60-4050-84a6-fff990fc0d3b None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:55.125995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1052/2086] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:50:55.133002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1035/2087] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:55.137574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c808b8e-4c5e-43e3-969e-219507b747a4 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:55.137876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c808b8e-4c5e-43e3-969e-219507b747a4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:55.138153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c808b8e-4c5e-43e3-969e-219507b747a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:55.146201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4838af7-0c3b-4a01-8e89-49ff03ca8679 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:55.152837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c4838af7-0c3b-4a01-8e89-49ff03ca8679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:55.165551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-8c808b8e-4c5e-43e3-969e-219507b747a4 None None] MFA Rules not processed for user `9ea1ee7219994d38a9002b3a1a46383e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:55.167097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8c808b8e-4c5e-43e3-969e-219507b747a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:55.217927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4838af7-0c3b-4a01-8e89-49ff03ca8679 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:55.222741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e41684-4f66-4c3b-8c84-5ea8eab5d6ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:55.223560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41684-4f66-4c3b-8c84-5ea8eab5d6ea service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:55.223843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41684-4f66-4c3b-8c84-5ea8eab5d6ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:55.224114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41684-4f66-4c3b-8c84-5ea8eab5d6ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:55.234725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e41684-4f66-4c3b-8c84-5ea8eab5d6ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:55.236177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e41684-4f66-4c3b-8c84-5ea8eab5d6ea service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:55.246309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1053/2088] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:50:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:55.248444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1036/2089] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:55.259012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c808b8e-4c5e-43e3-969e-219507b747a4 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:55.260555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8c808b8e-4c5e-43e3-969e-219507b747a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:55.280299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c808b8e-4c5e-43e3-969e-219507b747a4 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:55.286897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:55.287523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:55.287807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:55.288082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:55.299126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:55.300979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:55.326002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1054/2090] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:55.373376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:55.399955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878575c-d26a-4924-a4be-97c5a533ac26 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:55.404010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878575c-d26a-4924-a4be-97c5a533ac26 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:55.404010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878575c-d26a-4924-a4be-97c5a533ac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:55.436536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f878575c-d26a-4924-a4be-97c5a533ac26 None None] MFA Rules not processed for user `63ff0ea67ee64e14aafc5c51cad9684f`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:55.440178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f878575c-d26a-4924-a4be-97c5a533ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:55.453527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:55.533054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1037/2091] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:50:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:56.126676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:56.128580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:56.150615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e36ac-75da-439b-841a-1bccb2f2ece1 service placement] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:56.157452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:56.158257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:56.158668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:56.158922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] PATH_INFO: `/v3/projects/60c9a58027884a8e82f8c442d1a3474e` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:56.162675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c9a58027884a8e82f8c442d1a3474e)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:56.172573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:56.417908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c9a58027884a8e82f8c442d1a3474e', 'request_id': 'req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:56.430697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878575c-d26a-4924-a4be-97c5a533ac26 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:56.432980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f878575c-d26a-4924-a4be-97c5a533ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:56.453952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878575c-d26a-4924-a4be-97c5a533ac26 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:56.577847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:56.579772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:56.580961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:56.581307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] PATH_INFO: `/v3/users/f40dc2ffd2be451e83ff742cc903c581` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:56.587764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f40dc2ffd2be451e83ff742cc903c581)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:56.597299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:56.601434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c9a58027884a8e82f8c442d1a3474e', 'request_id': 'req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:56.612002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c9a58027884a8e82f8c442d1a3474e', 'request_id': 'req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:56.642456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] The token cache is being invalidate because project 60c9a58027884a8e82f8c442d1a3474e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:56.654110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:56.654110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:56.654110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1055/2092] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:56 2022] DELETE /identity/v3/projects/60c9a58027884a8e82f8c442d1a3474e => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:56.656912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:56.658229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:56.678862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff215f3-49cf-4cd6-aa33-6f41b506fdd6 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:56.689355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:56.690794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:56.691104 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:56.691625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] PATH_INFO: `/v3/users/40b559fef39d4a3a92fe222f1e138876` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:56.698302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b559fef39d4a3a92fe222f1e138876)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:56.700882 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:57.069380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b559fef39d4a3a92fe222f1e138876'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.083208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f40dc2ffd2be451e83ff742cc903c581'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.090223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b559fef39d4a3a92fe222f1e138876'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.201709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f40dc2ffd2be451e83ff742cc903c581'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.202373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f40dc2ffd2be451e83ff742cc903c581'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.210666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f40dc2ffd2be451e83ff742cc903c581'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.211364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f40dc2ffd2be451e83ff742cc903c581'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.228888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1038/2093] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:56 2022] DELETE /identity/v3/users/f40dc2ffd2be451e83ff742cc903c581 => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:57.289292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b559fef39d4a3a92fe222f1e138876'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.290151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b559fef39d4a3a92fe222f1e138876'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.290996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b559fef39d4a3a92fe222f1e138876'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:57.313709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1056/2094] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:56 2022] DELETE /identity/v3/users/40b559fef39d4a3a92fe222f1e138876 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:57.574106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:57.576950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:57.755867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:57.759208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ff1a8-81cf-49a4-9219-dc6a527411c9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:57.836378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:57.947865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:57.948901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:57.949374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:57.949816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] PATH_INFO: `/v3/projects/523a84ad16e54e2aa69d18a695345e2f` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:57.951711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=523a84ad16e54e2aa69d18a695345e2f)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:57.955280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:58.152076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523a84ad16e54e2aa69d18a695345e2f', 'request_id': 'req-27d75d3c-986c-4d60-98b9-edc8a7ef6294'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.294100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:58.297145 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:58.323427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49993a36-5d34-404f-ab36-fd8b7197d54a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:58.345833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523a84ad16e54e2aa69d18a695345e2f', 'request_id': 'req-27d75d3c-986c-4d60-98b9-edc8a7ef6294'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.366044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523a84ad16e54e2aa69d18a695345e2f', 'request_id': 'req-27d75d3c-986c-4d60-98b9-edc8a7ef6294'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.391636 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] The token cache is being invalidate because project 523a84ad16e54e2aa69d18a695345e2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:58.391952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.398498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.403045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1039/2095] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:57 2022] DELETE /identity/v3/projects/523a84ad16e54e2aa69d18a695345e2f => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:58.410339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:58.412193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:58.474581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:58.475241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:58.475500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:58.475780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] PATH_INFO: `/v3/projects/0ffdd0764f8b4726ab1e02c86aecbaf3` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:58.477002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ffdd0764f8b4726ab1e02c86aecbaf3)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:58.479128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:58.526500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d75d3c-986c-4d60-98b9-edc8a7ef6294 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:58.540720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:58.540720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:58.540720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:58.540720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] PATH_INFO: `/v3/users/0cea4e5d6a3c4d0ab0d26873682243a2` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:58.541345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cea4e5d6a3c4d0ab0d26873682243a2)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:58.541892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:58.680919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ffdd0764f8b4726ab1e02c86aecbaf3', 'request_id': 'req-35e7bf2d-e271-473a-a4c4-a92ef4d54608'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.752519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ffdd0764f8b4726ab1e02c86aecbaf3', 'request_id': 'req-35e7bf2d-e271-473a-a4c4-a92ef4d54608'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.772193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cea4e5d6a3c4d0ab0d26873682243a2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.779733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ffdd0764f8b4726ab1e02c86aecbaf3', 'request_id': 'req-35e7bf2d-e271-473a-a4c4-a92ef4d54608'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.793110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] The token cache is being invalidate because project 0ffdd0764f8b4726ab1e02c86aecbaf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:58.793331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.794642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.796790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1057/2096] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:58 2022] DELETE /identity/v3/projects/0ffdd0764f8b4726ab1e02c86aecbaf3 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:58.970971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cea4e5d6a3c4d0ab0d26873682243a2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:58.974882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cea4e5d6a3c4d0ab0d26873682243a2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:59.029659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cea4e5d6a3c4d0ab0d26873682243a2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:59.035255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7b0ba0ca-4eac-4944-9c40-af8f26974091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cea4e5d6a3c4d0ab0d26873682243a2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:59.164010 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1040/2097] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:58 2022] DELETE /identity/v3/users/0cea4e5d6a3c4d0ab0d26873682243a2 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:59.626071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:59.654137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:59.657077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:59.779581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7bf2d-e271-473a-a4c4-a92ef4d54608 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:59.874806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:59.875464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:59.875744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:59.876054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] PATH_INFO: `/v3/projects/9161e3d8c1214933a591db66c32d7c05` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:59.877292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9161e3d8c1214933a591db66c32d7c05)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:59.879325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:00.042369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9161e3d8c1214933a591db66c32d7c05', 'request_id': 'req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:00.253700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9161e3d8c1214933a591db66c32d7c05', 'request_id': 'req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:00.262920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9161e3d8c1214933a591db66c32d7c05', 'request_id': 'req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:00.278223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] The token cache is being invalidate because project 9161e3d8c1214933a591db66c32d7c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:00.278522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:00.279203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7b186a90-3bfc-44f4-a3fd-f62c1c4744a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:00.281075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1058/2098] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:59 2022] DELETE /identity/v3/projects/9161e3d8c1214933a591db66c32d7c05 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:00.288620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f323c-c0df-48ef-b3ae-c229002e4f7d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:00.289500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f323c-c0df-48ef-b3ae-c229002e4f7d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:00.290501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f323c-c0df-48ef-b3ae-c229002e4f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:00.342353 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-365f323c-c0df-48ef-b3ae-c229002e4f7d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:00.344184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-365f323c-c0df-48ef-b3ae-c229002e4f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:00.490595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1041/2099] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:00.492581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd07c-3870-4d72-8dee-a4413d06777b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:00.493138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd07c-3870-4d72-8dee-a4413d06777b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:00.493708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd07c-3870-4d72-8dee-a4413d06777b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:00.529896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-df6bd07c-3870-4d72-8dee-a4413d06777b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:00.531785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-df6bd07c-3870-4d72-8dee-a4413d06777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:00.542575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1059/2100] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:00.550092 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f323c-c0df-48ef-b3ae-c229002e4f7d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:00.566660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f323c-c0df-48ef-b3ae-c229002e4f7d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:00.574500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:00.575265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:00.576252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:00.576570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:00.578800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:00.579671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:00.581734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:00.602634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1042/2101] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:51:00.607213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bd07c-3870-4d72-8dee-a4413d06777b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:00.621546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bd07c-3870-4d72-8dee-a4413d06777b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:00.630595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dec2b-d5ee-47d9-82af-5cc5e1c7603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:00.631235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dec2b-d5ee-47d9-82af-5cc5e1c7603c admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:00.631523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dec2b-d5ee-47d9-82af-5cc5e1c7603c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:00.631807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dec2b-d5ee-47d9-82af-5cc5e1c7603c admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:00.632446 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dec2b-d5ee-47d9-82af-5cc5e1c7603c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:00.634829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dec2b-d5ee-47d9-82af-5cc5e1c7603c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:00.805495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1060/2102] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:00 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:00.808150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:00.819504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295fb12-fdd1-4255-bdcd-4217e10899e4 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:00.939502 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baecfe-686e-4f95-badf-c32e3a6905a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:00.940684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baecfe-686e-4f95-badf-c32e3a6905a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:00.941189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baecfe-686e-4f95-badf-c32e3a6905a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:00.941702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baecfe-686e-4f95-badf-c32e3a6905a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:00.943355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9baecfe-686e-4f95-badf-c32e3a6905a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:00.948604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9baecfe-686e-4f95-badf-c32e3a6905a2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:01.065979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:01.083611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1043/2103] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:00 2022] POST /identity/v3/users => generated 291 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:01.085635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dec2b-d5ee-47d9-82af-5cc5e1c7603c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:01.104092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dec2b-d5ee-47d9-82af-5cc5e1c7603c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:01.120186 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e811c1f-f9d8-454e-93fc-ee59e3b4a13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:01.120806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e811c1f-f9d8-454e-93fc-ee59e3b4a13e admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:01.121345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e811c1f-f9d8-454e-93fc-ee59e3b4a13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:01.121831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e811c1f-f9d8-454e-93fc-ee59e3b4a13e admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:01.123342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e811c1f-f9d8-454e-93fc-ee59e3b4a13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:01.127116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e811c1f-f9d8-454e-93fc-ee59e3b4a13e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:01.150195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baecfe-686e-4f95-badf-c32e3a6905a2 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:01.150850 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1061/2104] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:01.167586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baecfe-686e-4f95-badf-c32e3a6905a2 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:01.170340 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d5616-5994-4ea0-86ee-81984ee1fd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:01.171136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d5616-5994-4ea0-86ee-81984ee1fd81 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:01.171472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d5616-5994-4ea0-86ee-81984ee1fd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:01.171751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d5616-5994-4ea0-86ee-81984ee1fd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:01.172470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d5616-5994-4ea0-86ee-81984ee1fd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:01.174235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d5616-5994-4ea0-86ee-81984ee1fd81 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:01.191706 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1044/2105] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:01.197871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e811c1f-f9d8-454e-93fc-ee59e3b4a13e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:01.231182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e811c1f-f9d8-454e-93fc-ee59e3b4a13e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:01.244192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:01.246069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:01.246613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:01.247790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] PATH_INFO: `/v3/projects/06a9d3c3aacb406abbc73d77ff77bcf6/users/148b46da68004a95a97004eb7e5310c9/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:01.288335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a9d3c3aacb406abbc73d77ff77bcf6, user_id=148b46da68004a95a97004eb7e5310c9, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:01.288945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:01.404687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1062/2106] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:01 2022] PUT /identity/v3/projects/06a9d3c3aacb406abbc73d77ff77bcf6/users/148b46da68004a95a97004eb7e5310c9/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:01.417609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57be4c-95c5-4d40-8c81-adce21e45b20 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:01.418213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57be4c-95c5-4d40-8c81-adce21e45b20 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:01.418296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57be4c-95c5-4d40-8c81-adce21e45b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:01.473843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6a57be4c-95c5-4d40-8c81-adce21e45b20 None None] MFA Rules not processed for user `148b46da68004a95a97004eb7e5310c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:01.475116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6a57be4c-95c5-4d40-8c81-adce21e45b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:01.562715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1045/2107] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:01.575651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:01.578948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:01.684511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772f6a9-71c4-4842-b9ef-b4d1cf9dec89 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:01.689043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf8721-b7bc-4fc3-9623-bb010652dde0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:01.689963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf8721-b7bc-4fc3-9623-bb010652dde0 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:01.690461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf8721-b7bc-4fc3-9623-bb010652dde0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:01.690938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf8721-b7bc-4fc3-9623-bb010652dde0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:01.693503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:01.708427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf8721-b7bc-4fc3-9623-bb010652dde0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:01.710263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf8721-b7bc-4fc3-9623-bb010652dde0 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:01.727019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1063/2108] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:01.747659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57be4c-95c5-4d40-8c81-adce21e45b20 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:01.832254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57be4c-95c5-4d40-8c81-adce21e45b20 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:01.838810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:01.839713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:01.839713 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:01.840606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] PATH_INFO: `/v3/users/148b46da68004a95a97004eb7e5310c9` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:01.841833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=148b46da68004a95a97004eb7e5310c9)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:01.844165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:01.860160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:02.057498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148b46da68004a95a97004eb7e5310c9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.168954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148b46da68004a95a97004eb7e5310c9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.169734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148b46da68004a95a97004eb7e5310c9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.178659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148b46da68004a95a97004eb7e5310c9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.179496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148b46da68004a95a97004eb7e5310c9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.199106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1046/2109] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:01 2022] DELETE /identity/v3/users/148b46da68004a95a97004eb7e5310c9 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:02.203554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf672-4a70-4d5b-83d5-d8364ed01c46 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:02.203862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf672-4a70-4d5b-83d5-d8364ed01c46 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:02.204214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf672-4a70-4d5b-83d5-d8364ed01c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:02.224678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-31aaf672-4a70-4d5b-83d5-d8364ed01c46 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:02.226080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-31aaf672-4a70-4d5b-83d5-d8364ed01c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:02.325806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1064/2110] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:02.353214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:02.356080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:02.484118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eac098-7075-48f9-9ca1-6da8ec123dc5 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:02.489176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889f533-890b-4e8a-8e37-73e4a83fd67f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:02.490360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889f533-890b-4e8a-8e37-73e4a83fd67f service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:02.490720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889f533-890b-4e8a-8e37-73e4a83fd67f service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:02.491066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889f533-890b-4e8a-8e37-73e4a83fd67f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:02.502914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889f533-890b-4e8a-8e37-73e4a83fd67f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:02.504507 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889f533-890b-4e8a-8e37-73e4a83fd67f service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:02.515709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1047/2111] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:02.842536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:03.433121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaf672-4a70-4d5b-83d5-d8364ed01c46 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.437922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-31aaf672-4a70-4d5b-83d5-d8364ed01c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:03.480919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaf672-4a70-4d5b-83d5-d8364ed01c46 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.500881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.502210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.503327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.503783 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] PATH_INFO: `/v3/projects/06a9d3c3aacb406abbc73d77ff77bcf6` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.505301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a9d3c3aacb406abbc73d77ff77bcf6)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.508762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.537044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:03.700680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a9d3c3aacb406abbc73d77ff77bcf6', 'request_id': 'req-4e1ea659-7d9c-4108-a4ff-866a0069a348'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.795761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a9d3c3aacb406abbc73d77ff77bcf6', 'request_id': 'req-4e1ea659-7d9c-4108-a4ff-866a0069a348'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.804500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a9d3c3aacb406abbc73d77ff77bcf6', 'request_id': 'req-4e1ea659-7d9c-4108-a4ff-866a0069a348'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.819270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:03.821326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] The token cache is being invalidate because project 06a9d3c3aacb406abbc73d77ff77bcf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:03.822012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.823414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e1ea659-7d9c-4108-a4ff-866a0069a348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.826569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1065/2112] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:03 2022] DELETE /identity/v3/projects/06a9d3c3aacb406abbc73d77ff77bcf6 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:03.832164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5d691-6e93-4afa-83c2-f9f87651c4b8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.832679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5d691-6e93-4afa-83c2-f9f87651c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.833154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5d691-6e93-4afa-83c2-f9f87651c4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.856434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-beb5d691-6e93-4afa-83c2-f9f87651c4b8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:03.858208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-beb5d691-6e93-4afa-83c2-f9f87651c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:03.953611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1048/2113] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:03.963899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbd76-32e8-47f2-9c0a-dda9964e819c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.964247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbd76-32e8-47f2-9c0a-dda9964e819c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.964524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbd76-32e8-47f2-9c0a-dda9964e819c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.990985 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c86cbd76-32e8-47f2-9c0a-dda9964e819c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:03.992453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c86cbd76-32e8-47f2-9c0a-dda9964e819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:04.003397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1066/2114] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:04.007735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5d691-6e93-4afa-83c2-f9f87651c4b8 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.018699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5d691-6e93-4afa-83c2-f9f87651c4b8 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.024495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.025402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.025761 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.026148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.026928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:04.027329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.029305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.047945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1049/2115] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:04.060755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cbd76-32e8-47f2-9c0a-dda9964e819c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.071828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cbd76-32e8-47f2-9c0a-dda9964e819c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.078894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c69200-e287-4e63-a5ae-cd051f2c1a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.079683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c69200-e287-4e63-a5ae-cd051f2c1a40 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.080086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c69200-e287-4e63-a5ae-cd051f2c1a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.080558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c69200-e287-4e63-a5ae-cd051f2c1a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.081775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c69200-e287-4e63-a5ae-cd051f2c1a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.084045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c69200-e287-4e63-a5ae-cd051f2c1a40 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.188966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1067/2116] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:04 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:04.193342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.205616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153171-3f16-430a-a336-0bfb1a0888d1 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.270568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681e36-29f6-4ae8-adb7-ffb2ee6076c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.271190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681e36-29f6-4ae8-adb7-ffb2ee6076c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.271474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681e36-29f6-4ae8-adb7-ffb2ee6076c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.271750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681e36-29f6-4ae8-adb7-ffb2ee6076c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.272432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681e36-29f6-4ae8-adb7-ffb2ee6076c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.274465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681e36-29f6-4ae8-adb7-ffb2ee6076c9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.358439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1050/2117] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:04 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:04.362491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:04.363808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c69200-e287-4e63-a5ae-cd051f2c1a40 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.375664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c69200-e287-4e63-a5ae-cd051f2c1a40 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.381082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad48f4-5c52-43d0-907c-160d5fe130fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.381362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad48f4-5c52-43d0-907c-160d5fe130fb admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.381648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad48f4-5c52-43d0-907c-160d5fe130fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.381954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad48f4-5c52-43d0-907c-160d5fe130fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.382626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad48f4-5c52-43d0-907c-160d5fe130fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.384247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad48f4-5c52-43d0-907c-160d5fe130fb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.399851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1068/2118] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:04.401417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681e36-29f6-4ae8-adb7-ffb2ee6076c9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.413935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681e36-29f6-4ae8-adb7-ffb2ee6076c9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.418859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3110d0-87d5-4362-a4e7-c92684cd6ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.419463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3110d0-87d5-4362-a4e7-c92684cd6ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.419768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3110d0-87d5-4362-a4e7-c92684cd6ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.420045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3110d0-87d5-4362-a4e7-c92684cd6ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.420698 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3110d0-87d5-4362-a4e7-c92684cd6ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.422382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3110d0-87d5-4362-a4e7-c92684cd6ec0 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.433795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1051/2119] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:04.438029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad48f4-5c52-43d0-907c-160d5fe130fb admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.451957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad48f4-5c52-43d0-907c-160d5fe130fb admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.459588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.460418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.460903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.461380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] PATH_INFO: `/v3/projects/64041520ba8044f69004611b93ab7833/users/7f305273ae74431f879f9ee3c1914443/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.481967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64041520ba8044f69004611b93ab7833, user_id=7f305273ae74431f879f9ee3c1914443, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.485055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.556611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1069/2120] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:04 2022] PUT /identity/v3/projects/64041520ba8044f69004611b93ab7833/users/7f305273ae74431f879f9ee3c1914443/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:04.569050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac073f-fd67-4bbf-a6a1-9993a307cc81 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.570044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac073f-fd67-4bbf-a6a1-9993a307cc81 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.570790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac073f-fd67-4bbf-a6a1-9993a307cc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.578090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:04.610770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-61ac073f-fd67-4bbf-a6a1-9993a307cc81 None None] MFA Rules not processed for user `7f305273ae74431f879f9ee3c1914443`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:04.612543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-61ac073f-fd67-4bbf-a6a1-9993a307cc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:04.705903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1052/2121] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:04.717465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.719644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:04.798803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022c860-cc28-45d2-8ffe-8134b6bbe4f4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.803784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfe858-d059-4e38-8058-439c33934484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.804497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe858-d059-4e38-8058-439c33934484 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.804991 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe858-d059-4e38-8058-439c33934484 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.805275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe858-d059-4e38-8058-439c33934484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.817910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfe858-d059-4e38-8058-439c33934484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.819687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfe858-d059-4e38-8058-439c33934484 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.831133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1070/2122] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:04.859741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac073f-fd67-4bbf-a6a1-9993a307cc81 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.983808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac073f-fd67-4bbf-a6a1-9993a307cc81 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.991578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.992640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.993131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.993609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] PATH_INFO: `/v3/users/7f305273ae74431f879f9ee3c1914443` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.995594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f305273ae74431f879f9ee3c1914443)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.999445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:05.236829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f305273ae74431f879f9ee3c1914443'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:05.309354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f305273ae74431f879f9ee3c1914443'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:05.309933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f305273ae74431f879f9ee3c1914443'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:05.318581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f305273ae74431f879f9ee3c1914443'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:05.319296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f305273ae74431f879f9ee3c1914443'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:05.340472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1053/2123] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:04 2022] DELETE /identity/v3/users/7f305273ae74431f879f9ee3c1914443 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:05.347798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fad6-5147-4905-ae87-25a49947d7ee None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:05.348302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fad6-5147-4905-ae87-25a49947d7ee None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:05.348804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fad6-5147-4905-ae87-25a49947d7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:05.369660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f855fad6-5147-4905-ae87-25a49947d7ee None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:05.370928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f855fad6-5147-4905-ae87-25a49947d7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:05.455161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:05.458473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1071/2124] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:05.470991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:05.472043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:05.545184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c62911-8c80-4a60-9574-8bc34db2c6e0 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:05.550054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92000d-5840-41ee-b7fb-5f2042757970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:05.550706 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92000d-5840-41ee-b7fb-5f2042757970 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:05.550984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92000d-5840-41ee-b7fb-5f2042757970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:05.551255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92000d-5840-41ee-b7fb-5f2042757970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:05.565750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92000d-5840-41ee-b7fb-5f2042757970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:05.569940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92000d-5840-41ee-b7fb-5f2042757970 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:05.588150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1054/2125] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:05.817712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:06.251366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fad6-5147-4905-ae87-25a49947d7ee None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:06.254027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f855fad6-5147-4905-ae87-25a49947d7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:06.276363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fad6-5147-4905-ae87-25a49947d7ee None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:06.287153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:06.288170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:06.288568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:06.289020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] PATH_INFO: `/v3/projects/64041520ba8044f69004611b93ab7833` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:06.290538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=64041520ba8044f69004611b93ab7833)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:06.293212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:06.467521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64041520ba8044f69004611b93ab7833', 'request_id': 'req-2af303a7-db68-48da-bced-7bb714f7ecbb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:06.612822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64041520ba8044f69004611b93ab7833', 'request_id': 'req-2af303a7-db68-48da-bced-7bb714f7ecbb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:06.622282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64041520ba8044f69004611b93ab7833', 'request_id': 'req-2af303a7-db68-48da-bced-7bb714f7ecbb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:06.635435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] The token cache is being invalidate because project 64041520ba8044f69004611b93ab7833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:06.636038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:06.636834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2af303a7-db68-48da-bced-7bb714f7ecbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:06.640204 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1072/2126] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:06 2022] DELETE /identity/v3/projects/64041520ba8044f69004611b93ab7833 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:06.647498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86151674-5210-4722-9cd7-20c4674163f9 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:06.648529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86151674-5210-4722-9cd7-20c4674163f9 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:06.648950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86151674-5210-4722-9cd7-20c4674163f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:06.683935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-86151674-5210-4722-9cd7-20c4674163f9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:06.685532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-86151674-5210-4722-9cd7-20c4674163f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:06.792093 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1055/2127] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:06.800512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd90ab8-24b5-4469-9778-baca8ac91958 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:06.800797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd90ab8-24b5-4469-9778-baca8ac91958 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:06.801586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd90ab8-24b5-4469-9778-baca8ac91958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:06.835351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-cdd90ab8-24b5-4469-9778-baca8ac91958 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:06.836603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cdd90ab8-24b5-4469-9778-baca8ac91958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:06.845869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1073/2128] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:06.850188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86151674-5210-4722-9cd7-20c4674163f9 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:06.861981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86151674-5210-4722-9cd7-20c4674163f9 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:06.867710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:06.868343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:06.868616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:06.868885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:06.869547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:06.869883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:06.871557 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:06.886921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1056/2129] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:06.891810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd90ab8-24b5-4469-9778-baca8ac91958 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:06.906734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd90ab8-24b5-4469-9778-baca8ac91958 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:06.911840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986ef8b-e410-4405-89d9-56261800805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:06.912704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ef8b-e410-4405-89d9-56261800805d admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:06.913165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ef8b-e410-4405-89d9-56261800805d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:06.913584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ef8b-e410-4405-89d9-56261800805d admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:06.914504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986ef8b-e410-4405-89d9-56261800805d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:06.916594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986ef8b-e410-4405-89d9-56261800805d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:07.052250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1074/2130] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:06 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:07.056578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:07.064193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:07.072117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e750a-85df-4af7-a513-6785bc358c4b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:07.136715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdacac-2705-479e-afa7-e10e57e8cf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:07.137872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdacac-2705-479e-afa7-e10e57e8cf11 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:07.137872 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdacac-2705-479e-afa7-e10e57e8cf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:07.138130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdacac-2705-479e-afa7-e10e57e8cf11 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:07.138880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdacac-2705-479e-afa7-e10e57e8cf11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:07.141232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdacac-2705-479e-afa7-e10e57e8cf11 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:07.279666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1057/2131] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:07 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:07.284801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986ef8b-e410-4405-89d9-56261800805d admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:07.300659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986ef8b-e410-4405-89d9-56261800805d admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:07.307018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ac6bc-9e9b-4c2e-8687-268afdf01e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:07.308064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ac6bc-9e9b-4c2e-8687-268afdf01e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:07.308347 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ac6bc-9e9b-4c2e-8687-268afdf01e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:07.311207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ac6bc-9e9b-4c2e-8687-268afdf01e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:07.311207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ac6bc-9e9b-4c2e-8687-268afdf01e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:07.311510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ac6bc-9e9b-4c2e-8687-268afdf01e8a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:07.325119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1075/2132] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:07.329608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdacac-2705-479e-afa7-e10e57e8cf11 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:07.340642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdacac-2705-479e-afa7-e10e57e8cf11 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:07.352047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b73ae-587b-45f2-9e63-0ea957718aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:07.352737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b73ae-587b-45f2-9e63-0ea957718aec admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:07.353584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b73ae-587b-45f2-9e63-0ea957718aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:07.354130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b73ae-587b-45f2-9e63-0ea957718aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:07.355260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b73ae-587b-45f2-9e63-0ea957718aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:07.357023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b73ae-587b-45f2-9e63-0ea957718aec admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:07.386444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1058/2133] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:07.390775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ac6bc-9e9b-4c2e-8687-268afdf01e8a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:07.410234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ac6bc-9e9b-4c2e-8687-268afdf01e8a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:07.422760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:07.423634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:07.424122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:07.424562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] PATH_INFO: `/v3/projects/a770f0caef364853a4033422b76ab918/users/bfdf1fd67f954cde9ad1334bb93ec4e6/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:07.450394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a770f0caef364853a4033422b76ab918, user_id=bfdf1fd67f954cde9ad1334bb93ec4e6, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:07.454165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:07.583312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1076/2134] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:07 2022] PUT /identity/v3/projects/a770f0caef364853a4033422b76ab918/users/bfdf1fd67f954cde9ad1334bb93ec4e6/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:07.591524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b15a6-9e46-4c37-be4b-c9bafa28074d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:07.591911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b15a6-9e46-4c37-be4b-c9bafa28074d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:07.592288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b15a6-9e46-4c37-be4b-c9bafa28074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:07.636078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-151b15a6-9e46-4c37-be4b-c9bafa28074d None None] MFA Rules not processed for user `bfdf1fd67f954cde9ad1334bb93ec4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:07.636078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-151b15a6-9e46-4c37-be4b-c9bafa28074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:07.759378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1059/2135] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:07.842702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:07.844908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:07.925574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf22473-68f9-4b4c-8e87-32ed5983c0e9 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:07.929393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:07.930081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:07.930546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:07.931383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:07.940018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:07.951016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:07.953430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:07.964473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1077/2136] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:10.136967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef2695-a18d-4a48-b004-da9956f8b9c4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.137456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef2695-a18d-4a48-b004-da9956f8b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.137549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef2695-a18d-4a48-b004-da9956f8b9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.157779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a5ef2695-a18d-4a48-b004-da9956f8b9c4 None None] MFA Rules not processed for user `bfdf1fd67f954cde9ad1334bb93ec4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:10.159075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a5ef2695-a18d-4a48-b004-da9956f8b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:10.166102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1060/2137] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:10.918125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:12.367787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:15.431966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:17.494404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:17.496321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:17.581921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecdcca-7ee6-4d1e-8f54-f3313de14f72 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:17.586722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:17.587382 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:17.587671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:17.588020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] PATH_INFO: `/v3/users/1c65965f8254439d94acc81461d443d6` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:17.589339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c65965f8254439d94acc81461d443d6)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:17.591347 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:18.040335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c65965f8254439d94acc81461d443d6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.076181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c65965f8254439d94acc81461d443d6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.162425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c65965f8254439d94acc81461d443d6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.163355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c65965f8254439d94acc81461d443d6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.164568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c65965f8254439d94acc81461d443d6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.184771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1078/2138] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:17 2022] DELETE /identity/v3/users/1c65965f8254439d94acc81461d443d6 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:18.193122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f12e7-0d3d-4762-8a90-cb794c45f17d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:18.193656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f12e7-0d3d-4762-8a90-cb794c45f17d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:18.194554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f12e7-0d3d-4762-8a90-cb794c45f17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:18.230707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-289f12e7-0d3d-4762-8a90-cb794c45f17d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:18.236729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-289f12e7-0d3d-4762-8a90-cb794c45f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:18.348136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1061/2139] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:18.399258 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:18.401481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:18.489665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:18.504372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053b15a-0adc-41c0-a89c-4eefe2c5b668 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:18.508679 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4537c1-828a-405a-a2ad-fb71118c5bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:18.509446 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4537c1-828a-405a-a2ad-fb71118c5bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:18.509801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4537c1-828a-405a-a2ad-fb71118c5bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:18.514303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4537c1-828a-405a-a2ad-fb71118c5bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:18.523131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4537c1-828a-405a-a2ad-fb71118c5bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:18.525129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4537c1-828a-405a-a2ad-fb71118c5bd9 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:18.544755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1079/2140] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:19.177551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f12e7-0d3d-4762-8a90-cb794c45f17d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.179088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-289f12e7-0d3d-4762-8a90-cb794c45f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:19.192893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f12e7-0d3d-4762-8a90-cb794c45f17d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.198370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.199843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.200031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.200239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] PATH_INFO: `/v3/projects/ebd39a61abf04a2f9c344a81e83eabb2` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.203017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd39a61abf04a2f9c344a81e83eabb2)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.205332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.370431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:19.382138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd39a61abf04a2f9c344a81e83eabb2', 'request_id': 'req-78788bb6-d958-4ee3-bdf2-bb681b296efe'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.489836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd39a61abf04a2f9c344a81e83eabb2', 'request_id': 'req-78788bb6-d958-4ee3-bdf2-bb681b296efe'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.497520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd39a61abf04a2f9c344a81e83eabb2', 'request_id': 'req-78788bb6-d958-4ee3-bdf2-bb681b296efe'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.518540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] The token cache is being invalidate because project ebd39a61abf04a2f9c344a81e83eabb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:19.518834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.519516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.521095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1062/2141] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:19 2022] DELETE /identity/v3/projects/ebd39a61abf04a2f9c344a81e83eabb2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:19.525575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4537c1-828a-405a-a2ad-fb71118c5bd9 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.537306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4537c1-828a-405a-a2ad-fb71118c5bd9 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.641966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.643231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.643523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.643995 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] PATH_INFO: `/v3/users/601c45ad63ec4938b1f79deade2e54e7` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.645444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=601c45ad63ec4938b1f79deade2e54e7)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.647936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.896492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601c45ad63ec4938b1f79deade2e54e7'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.923450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601c45ad63ec4938b1f79deade2e54e7'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:20.076666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601c45ad63ec4938b1f79deade2e54e7'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:20.078495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601c45ad63ec4938b1f79deade2e54e7'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:20.079666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ba54aea6-2cb7-4dc0-a3f3-a73494beeff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601c45ad63ec4938b1f79deade2e54e7'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:20.114891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1080/2142] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:19 2022] DELETE /identity/v3/users/601c45ad63ec4938b1f79deade2e54e7 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:20.845444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:20.846972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:20.861754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78788bb6-d958-4ee3-bdf2-bb681b296efe admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:20.903776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9deec-9c27-4f91-b937-13ca94a6c16c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.906212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9deec-9c27-4f91-b937-13ca94a6c16c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.906684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9deec-9c27-4f91-b937-13ca94a6c16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.946934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-21d9deec-9c27-4f91-b937-13ca94a6c16c None None] MFA Rules not processed for user `251bc85f89774142a97acd77895b14e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:20.948993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-21d9deec-9c27-4f91-b937-13ca94a6c16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:20.971783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:20.972858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.973317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.973764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] PATH_INFO: `/v3/projects/c05acfb001f0476d906f99461b9121e3` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.975970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c05acfb001f0476d906f99461b9121e3)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.979456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:21.046421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1081/2143] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:21.156082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c05acfb001f0476d906f99461b9121e3', 'request_id': 'req-645e02bc-d091-4dcc-9a40-521b60e7d11d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:21.258769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c05acfb001f0476d906f99461b9121e3', 'request_id': 'req-645e02bc-d091-4dcc-9a40-521b60e7d11d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:21.269847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c05acfb001f0476d906f99461b9121e3', 'request_id': 'req-645e02bc-d091-4dcc-9a40-521b60e7d11d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:21.294638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] The token cache is being invalidate because project c05acfb001f0476d906f99461b9121e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:21.299576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:21.299576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-645e02bc-d091-4dcc-9a40-521b60e7d11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:21.301490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1063/2144] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:20 2022] DELETE /identity/v3/projects/c05acfb001f0476d906f99461b9121e3 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:21.307997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503a43-85fa-4f15-ab25-c3a5027905fc None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:21.308514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503a43-85fa-4f15-ab25-c3a5027905fc None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:21.309019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503a43-85fa-4f15-ab25-c3a5027905fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:21.344541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-27503a43-85fa-4f15-ab25-c3a5027905fc None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:21.346449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-27503a43-85fa-4f15-ab25-c3a5027905fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:21.355202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:21.500505 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1082/2145] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:21.508168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42247-d699-4ba2-b0cd-8dfda3fbd361 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:21.508783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42247-d699-4ba2-b0cd-8dfda3fbd361 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:21.509052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42247-d699-4ba2-b0cd-8dfda3fbd361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:21.539447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6ca42247-d699-4ba2-b0cd-8dfda3fbd361 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:21.540837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6ca42247-d699-4ba2-b0cd-8dfda3fbd361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:21.571457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1064/2146] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:21.576027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27503a43-85fa-4f15-ab25-c3a5027905fc None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:21.588223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27503a43-85fa-4f15-ab25-c3a5027905fc None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:21.596072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:21.596827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:21.597192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:21.597536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:21.598269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:21.598675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:21.600409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:21.626910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1083/2147] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:21.627925 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42247-d699-4ba2-b0cd-8dfda3fbd361 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:21.646737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42247-d699-4ba2-b0cd-8dfda3fbd361 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:21.656030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912588b8-5a01-4522-950d-36fe659b51f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:21.657426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912588b8-5a01-4522-950d-36fe659b51f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:21.658186 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912588b8-5a01-4522-950d-36fe659b51f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:21.658649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912588b8-5a01-4522-950d-36fe659b51f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:21.659810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912588b8-5a01-4522-950d-36fe659b51f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:21.663212 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912588b8-5a01-4522-950d-36fe659b51f5 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:21.825568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1065/2148] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:21 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:21.826973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:21.840795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cd1b0-f2a9-4532-9035-e55523975d27 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:21.938032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180140e5-6744-4d1c-b6bb-14c4c8213d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:21.939031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180140e5-6744-4d1c-b6bb-14c4c8213d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:21.939031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180140e5-6744-4d1c-b6bb-14c4c8213d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:21.943707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180140e5-6744-4d1c-b6bb-14c4c8213d90 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:21.943707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180140e5-6744-4d1c-b6bb-14c4c8213d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:21.943707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180140e5-6744-4d1c-b6bb-14c4c8213d90 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:22.039517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1084/2149] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:21 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:22.041084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912588b8-5a01-4522-950d-36fe659b51f5 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:22.053254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912588b8-5a01-4522-950d-36fe659b51f5 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:22.059736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e2d92-e782-4b8b-a6c2-ab2e9607db6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:22.060510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e2d92-e782-4b8b-a6c2-ab2e9607db6d admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:22.064043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e2d92-e782-4b8b-a6c2-ab2e9607db6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:22.064043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e2d92-e782-4b8b-a6c2-ab2e9607db6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:22.064043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e2d92-e782-4b8b-a6c2-ab2e9607db6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:22.064935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e2d92-e782-4b8b-a6c2-ab2e9607db6d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:22.083240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1066/2150] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:22.084457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180140e5-6744-4d1c-b6bb-14c4c8213d90 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:22.098565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180140e5-6744-4d1c-b6bb-14c4c8213d90 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:22.104151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4b40-e128-44ff-ad85-ebc3f0fff62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:22.104782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4b40-e128-44ff-ad85-ebc3f0fff62e admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:22.105061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4b40-e128-44ff-ad85-ebc3f0fff62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:22.105366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4b40-e128-44ff-ad85-ebc3f0fff62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:22.106085 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4b40-e128-44ff-ad85-ebc3f0fff62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:22.108276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4b40-e128-44ff-ad85-ebc3f0fff62e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:22.141758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e2d92-e782-4b8b-a6c2-ab2e9607db6d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:22.148083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1085/2151] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:22.173940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e2d92-e782-4b8b-a6c2-ab2e9607db6d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:22.189764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6305b-06c2-40b4-b39f-ed9b492f46d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:22.190932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6305b-06c2-40b4-b39f-ed9b492f46d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:22.191226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6305b-06c2-40b4-b39f-ed9b492f46d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:22.191552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6305b-06c2-40b4-b39f-ed9b492f46d8 admin admin] PATH_INFO: `/v3/projects/bb33707df06144029cb726f5fdab0370/users/9c4902609ef84c6ca2cbddf03e83df7e/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:22.207580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:22.212474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6305b-06c2-40b4-b39f-ed9b492f46d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb33707df06144029cb726f5fdab0370, user_id=9c4902609ef84c6ca2cbddf03e83df7e, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:22.215157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6305b-06c2-40b4-b39f-ed9b492f46d8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:22.314058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1067/2152] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:22 2022] PUT /identity/v3/projects/bb33707df06144029cb726f5fdab0370/users/9c4902609ef84c6ca2cbddf03e83df7e/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:22.336266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394be2-3823-43aa-affb-58d41b748f30 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:22.336730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394be2-3823-43aa-affb-58d41b748f30 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:22.337197 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394be2-3823-43aa-affb-58d41b748f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:22.416340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c8394be2-3823-43aa-affb-58d41b748f30 None None] MFA Rules not processed for user `9c4902609ef84c6ca2cbddf03e83df7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:22.419358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c8394be2-3823-43aa-affb-58d41b748f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:22.529058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1086/2153] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:22.535005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6305b-06c2-40b4-b39f-ed9b492f46d8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:22.635197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6305b-06c2-40b4-b39f-ed9b492f46d8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:22.642821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:22.643723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:22.644558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:22.644980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:22.645743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:22.647841 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:22.795746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1068/2154] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:22 2022] POST /identity/v3/projects => generated 365 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:22.801623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8394be2-3823-43aa-affb-58d41b748f30 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:22.808689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:22.813411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:22.815168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8394be2-3823-43aa-affb-58d41b748f30 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:22.888173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee1a0e-a5a9-4cc9-a173-733770a37fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:22.889552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee1a0e-a5a9-4cc9-a173-733770a37fea admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:22.889932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee1a0e-a5a9-4cc9-a173-733770a37fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:22.890396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee1a0e-a5a9-4cc9-a173-733770a37fea admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:22.891089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee1a0e-a5a9-4cc9-a173-733770a37fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:22.893048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee1a0e-a5a9-4cc9-a173-733770a37fea admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:22.924134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfd1b2-d373-437f-8b3c-d0ddca0f886b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:22.932199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:22.933249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:22.933743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:22.934295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] PATH_INFO: `/v3/users/3a571a816c7141a4a3f49be082c47fca` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:22.937114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a571a816c7141a4a3f49be082c47fca)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:22.941272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:23.100850 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1087/2155] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:22 2022] POST /identity/v3/users => generated 289 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:23.106451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee1a0e-a5a9-4cc9-a173-733770a37fea admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:23.147673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee1a0e-a5a9-4cc9-a173-733770a37fea admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:23.155654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485e573-99a0-4e4f-96ad-f84d41342c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:23.157054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485e573-99a0-4e4f-96ad-f84d41342c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:23.157633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485e573-99a0-4e4f-96ad-f84d41342c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:23.158290 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485e573-99a0-4e4f-96ad-f84d41342c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:23.159623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485e573-99a0-4e4f-96ad-f84d41342c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:23.163000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485e573-99a0-4e4f-96ad-f84d41342c6f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:23.183151 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1088/2156] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:23.191368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485e573-99a0-4e4f-96ad-f84d41342c6f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:23.213495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485e573-99a0-4e4f-96ad-f84d41342c6f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:23.219869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:23.220698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:23.221091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:23.221432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] PATH_INFO: `/v3/projects/e3b839c09d4b4f108f3d1c0b5bf5efb2/users/c7bef72efe334fb89a7be65a8f698b5c/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:23.250680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b839c09d4b4f108f3d1c0b5bf5efb2, user_id=c7bef72efe334fb89a7be65a8f698b5c, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:23.254003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a571a816c7141a4a3f49be082c47fca'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:23.256978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:23.345726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a571a816c7141a4a3f49be082c47fca'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:23.346585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a571a816c7141a4a3f49be082c47fca'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:23.352414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1089/2157] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:23 2022] PUT /identity/v3/projects/e3b839c09d4b4f108f3d1c0b5bf5efb2/users/c7bef72efe334fb89a7be65a8f698b5c/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:23.356691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a571a816c7141a4a3f49be082c47fca'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:23.358589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:23.358975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-64825c90-d3fd-4a0f-b1d3-66b5b253cf00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a571a816c7141a4a3f49be082c47fca'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:23.361513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:23.380951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1069/2158] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:22 2022] DELETE /identity/v3/users/3a571a816c7141a4a3f49be082c47fca => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:23.387113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675e97-312f-4ec6-aefe-b5dd283d0d59 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:23.387742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675e97-312f-4ec6-aefe-b5dd283d0d59 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:23.388251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675e97-312f-4ec6-aefe-b5dd283d0d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:23.431952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-5f675e97-312f-4ec6-aefe-b5dd283d0d59 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:23.437643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5f675e97-312f-4ec6-aefe-b5dd283d0d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:23.477831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2cda7-59d3-428f-9bb2-c667a8bc4cbc admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:23.590191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1070/2159] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:23.604891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f675e97-312f-4ec6-aefe-b5dd283d0d59 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:23.612319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5f675e97-312f-4ec6-aefe-b5dd283d0d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:23.654549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84478c-2fc8-4aeb-924d-8fe073caf1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:23.657257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84478c-2fc8-4aeb-924d-8fe073caf1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:23.657779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84478c-2fc8-4aeb-924d-8fe073caf1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:23.658649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84478c-2fc8-4aeb-924d-8fe073caf1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:23.659388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84478c-2fc8-4aeb-924d-8fe073caf1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:23.662221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84478c-2fc8-4aeb-924d-8fe073caf1b5 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:23.694095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1090/2160] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:23.695802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84478c-2fc8-4aeb-924d-8fe073caf1b5 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:23.726482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84478c-2fc8-4aeb-924d-8fe073caf1b5 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:23.726482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:23.730440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797d492-01eb-4700-abcd-d8396cd96f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:23.731309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797d492-01eb-4700-abcd-d8396cd96f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:23.731836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797d492-01eb-4700-abcd-d8396cd96f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:23.732981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797d492-01eb-4700-abcd-d8396cd96f81 admin admin] PATH_INFO: `/v3/projects/e3b839c09d4b4f108f3d1c0b5bf5efb2/users/c7bef72efe334fb89a7be65a8f698b5c/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:23.736703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797d492-01eb-4700-abcd-d8396cd96f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b839c09d4b4f108f3d1c0b5bf5efb2, user_id=c7bef72efe334fb89a7be65a8f698b5c, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:23.746818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797d492-01eb-4700-abcd-d8396cd96f81 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:23.753691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f675e97-312f-4ec6-aefe-b5dd283d0d59 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:23.762627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:23.764793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:23.774084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:23.774084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:23.797424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:23.806339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1091/2161] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:23 2022] PUT /identity/v3/projects/e3b839c09d4b4f108f3d1c0b5bf5efb2/users/c7bef72efe334fb89a7be65a8f698b5c/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:23.806339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44a9e-5884-4f05-8d56-1e3df15d1c04 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:23.806339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44a9e-5884-4f05-8d56-1e3df15d1c04 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:23.806339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44a9e-5884-4f05-8d56-1e3df15d1c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:23.808724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:23.848553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1071/2162] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:23.868951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ddb44a9e-5884-4f05-8d56-1e3df15d1c04 None None] MFA Rules not processed for user `c7bef72efe334fb89a7be65a8f698b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:23.871642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ddb44a9e-5884-4f05-8d56-1e3df15d1c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:24.064943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1092/2163] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:24.083972 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:24.086734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:24.199447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34804a8c-27ee-45c7-96f0-5e6299252233 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:24.272269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:24.272950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:24.273250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:24.273542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:24.291989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:24.306079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:24.319793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1072/2164] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:24.819905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb44a9e-5884-4f05-8d56-1e3df15d1c04 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:24.821284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ddb44a9e-5884-4f05-8d56-1e3df15d1c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:24.840672 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:24.847303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:24.868541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45db1e4-b816-4bd4-92fd-e275f57cf586 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:24.872811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99146835-9771-4f5f-a355-8b5a99b7ef00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:24.876401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99146835-9771-4f5f-a355-8b5a99b7ef00 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:24.876401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99146835-9771-4f5f-a355-8b5a99b7ef00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:24.876401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99146835-9771-4f5f-a355-8b5a99b7ef00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:24.879031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-99146835-9771-4f5f-a355-8b5a99b7ef00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:24.938968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb44a9e-5884-4f05-8d56-1e3df15d1c04 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:24.944009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:24.948485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:24.948485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:24.948485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] PATH_INFO: `/v3/projects/5ba927bd8fd242eeb9968bea40e7c19c` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:24.948485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba927bd8fd242eeb9968bea40e7c19c)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:24.950259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:24.963173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99146835-9771-4f5f-a355-8b5a99b7ef00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:24.964863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99146835-9771-4f5f-a355-8b5a99b7ef00 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:24.980974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1073/2165] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:25.097394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba927bd8fd242eeb9968bea40e7c19c', 'request_id': 'req-d91a1675-3a43-4cbd-8798-0becebc090dc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.213811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba927bd8fd242eeb9968bea40e7c19c', 'request_id': 'req-d91a1675-3a43-4cbd-8798-0becebc090dc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.229731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba927bd8fd242eeb9968bea40e7c19c', 'request_id': 'req-d91a1675-3a43-4cbd-8798-0becebc090dc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.264713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] The token cache is being invalidate because project 5ba927bd8fd242eeb9968bea40e7c19c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:25.264713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.264713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d91a1675-3a43-4cbd-8798-0becebc090dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.264713 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1093/2166] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:24 2022] DELETE /identity/v3/projects/5ba927bd8fd242eeb9968bea40e7c19c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:25.265355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56a8fe-4b8d-4d91-b148-818e2cb1d705 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.265355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56a8fe-4b8d-4d91-b148-818e2cb1d705 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.265355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56a8fe-4b8d-4d91-b148-818e2cb1d705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.285941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6e56a8fe-4b8d-4d91-b148-818e2cb1d705 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:25.287164 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6e56a8fe-4b8d-4d91-b148-818e2cb1d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:25.379854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1074/2167] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:25.384080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:25.388216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f024-8587-415c-be32-18aaccd54914 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.389063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f024-8587-415c-be32-18aaccd54914 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.389977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f024-8587-415c-be32-18aaccd54914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.424245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-5f58f024-8587-415c-be32-18aaccd54914 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:25.425485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5f58f024-8587-415c-be32-18aaccd54914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:25.442108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1094/2168] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:25.443543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56a8fe-4b8d-4d91-b148-818e2cb1d705 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.454997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56a8fe-4b8d-4d91-b148-818e2cb1d705 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.466066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.466066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.466066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.466066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.466471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:25.466471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.470945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.485393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1075/2169] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:25.490598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58f024-8587-415c-be32-18aaccd54914 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.502271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58f024-8587-415c-be32-18aaccd54914 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.507269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a117b-3123-4cca-b612-6b3da4357ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.507950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a117b-3123-4cca-b612-6b3da4357ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.508246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a117b-3123-4cca-b612-6b3da4357ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.508522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a117b-3123-4cca-b612-6b3da4357ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.509181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a117b-3123-4cca-b612-6b3da4357ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.511148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a117b-3123-4cca-b612-6b3da4357ef7 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.632584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1095/2170] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:25.639014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.656613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16351a-ab7e-43f9-bf70-09fc21538c82 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.784651 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef276a01-4da6-45fe-836c-da4478e16d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.785624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276a01-4da6-45fe-836c-da4478e16d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.786106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276a01-4da6-45fe-836c-da4478e16d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.786574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276a01-4da6-45fe-836c-da4478e16d0a admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.787729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef276a01-4da6-45fe-836c-da4478e16d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.792351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef276a01-4da6-45fe-836c-da4478e16d0a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.886907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1076/2171] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:25.891620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a117b-3123-4cca-b612-6b3da4357ef7 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.903125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a117b-3123-4cca-b612-6b3da4357ef7 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.909615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587b003-cb4c-4351-9871-adea08522f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.910227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b003-cb4c-4351-9871-adea08522f00 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.910515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b003-cb4c-4351-9871-adea08522f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.910789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b003-cb4c-4351-9871-adea08522f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.911659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587b003-cb4c-4351-9871-adea08522f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.913868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587b003-cb4c-4351-9871-adea08522f00 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.927112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1096/2172] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:25.931418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef276a01-4da6-45fe-836c-da4478e16d0a admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.943973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef276a01-4da6-45fe-836c-da4478e16d0a admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.955484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a726ab-1bfb-4b73-aa95-38c92d67702b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.955484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a726ab-1bfb-4b73-aa95-38c92d67702b admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.955484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a726ab-1bfb-4b73-aa95-38c92d67702b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.955484 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a726ab-1bfb-4b73-aa95-38c92d67702b admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.956101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a726ab-1bfb-4b73-aa95-38c92d67702b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.956101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a726ab-1bfb-4b73-aa95-38c92d67702b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.969179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1077/2173] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:25.974566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587b003-cb4c-4351-9871-adea08522f00 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.986455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587b003-cb4c-4351-9871-adea08522f00 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.995796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.996468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.996751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.997020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] PATH_INFO: `/v3/projects/4fa9fd072c704af2950a9bb4339d374d/users/7ca82f3632bc404691835a2db6167939/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:26.018000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa9fd072c704af2950a9bb4339d374d, user_id=7ca82f3632bc404691835a2db6167939, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:26.020859 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:26.145432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1097/2174] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:25 2022] PUT /identity/v3/projects/4fa9fd072c704af2950a9bb4339d374d/users/7ca82f3632bc404691835a2db6167939/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:26.150663 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2986a-6e81-4f71-ba5c-7e317bb04ed8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:26.150953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2986a-6e81-4f71-ba5c-7e317bb04ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:26.151228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2986a-6e81-4f71-ba5c-7e317bb04ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:26.174822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b7d2986a-6e81-4f71-ba5c-7e317bb04ed8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:26.176041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b7d2986a-6e81-4f71-ba5c-7e317bb04ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:26.219332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:26.256460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1078/2175] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:26.270795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:26.272582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:26.343997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb43d3-3d2c-4981-b26e-6481d7c560c0 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:26.347960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:26.348581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:26.348863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:26.349140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:26.359360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:26.361086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:26.372288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1098/2176] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:28.121735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:28.144424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:28.570496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a19b6-3c73-4e47-926e-281c486dd71f None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:28.571109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a19b6-3c73-4e47-926e-281c486dd71f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:28.571602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a19b6-3c73-4e47-926e-281c486dd71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:28.614477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e13a19b6-3c73-4e47-926e-281c486dd71f None None] MFA Rules not processed for user `7ca82f3632bc404691835a2db6167939`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:28.615974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e13a19b6-3c73-4e47-926e-281c486dd71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:28.693886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1079/2177] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:28.707072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:28.708541 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:28.841380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e47b8-3947-46fe-a26f-dc2bb45be9aa service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:28.848330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:28.850226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:28.850803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:28.851330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:28.866602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:28.869379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:28.885949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1099/2178] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:29.417730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:30.818466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdec671-a314-4cc0-a7fc-72b06441b377 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.819148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdec671-a314-4cc0-a7fc-72b06441b377 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.819148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdec671-a314-4cc0-a7fc-72b06441b377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.842551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2cdec671-a314-4cc0-a7fc-72b06441b377 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:30.844044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2cdec671-a314-4cc0-a7fc-72b06441b377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:30.945665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1080/2179] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:30.995895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.997333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:31.050684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:31.071673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88bb8a-3047-409b-ab3b-c5aaa9a4417f service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:31.078132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7713b-9906-4ee5-a6ce-d4dc368e12ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:31.078617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7713b-9906-4ee5-a6ce-d4dc368e12ce service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:31.078906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7713b-9906-4ee5-a6ce-d4dc368e12ce service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:31.079181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7713b-9906-4ee5-a6ce-d4dc368e12ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:31.090670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7713b-9906-4ee5-a6ce-d4dc368e12ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:31.092302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7713b-9906-4ee5-a6ce-d4dc368e12ce service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:31.101451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1100/2180] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:51:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:51:34.672118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83641231-4071-4030-a509-0dda76ccd844 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.673502 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83641231-4071-4030-a509-0dda76ccd844 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.674051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83641231-4071-4030-a509-0dda76ccd844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.702201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-83641231-4071-4030-a509-0dda76ccd844 None None] MFA Rules not processed for user `7ca82f3632bc404691835a2db6167939`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:34.703793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-83641231-4071-4030-a509-0dda76ccd844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.712230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1081/2181] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:36.359906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b533-7791-454f-a04f-de1ff84250a1 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:36.360554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b533-7791-454f-a04f-de1ff84250a1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:36.361192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b533-7791-454f-a04f-de1ff84250a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:36.419408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3e05b533-7791-454f-a04f-de1ff84250a1 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:36.423098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3e05b533-7791-454f-a04f-de1ff84250a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:36.443099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1101/2182] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:51:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:36.519278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83641231-4071-4030-a509-0dda76ccd844 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:36.546103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83641231-4071-4030-a509-0dda76ccd844 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:36.563712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac24013-ae3a-4479-ad09-320d1f1c4b8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:36.567929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac24013-ae3a-4479-ad09-320d1f1c4b8b service swift] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:36.568846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac24013-ae3a-4479-ad09-320d1f1c4b8b service swift] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:36.570037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac24013-ae3a-4479-ad09-320d1f1c4b8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:36.600128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac24013-ae3a-4479-ad09-320d1f1c4b8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:36.602105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac24013-ae3a-4479-ad09-320d1f1c4b8b service swift] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:36.613217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1082/2183] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:51:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:51:38.589662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7dba3-d0a5-4275-9c30-52953b5e5ae7 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:38.590379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7dba3-d0a5-4275-9c30-52953b5e5ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:38.591676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7dba3-d0a5-4275-9c30-52953b5e5ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:38.629388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-22d7dba3-d0a5-4275-9c30-52953b5e5ae7 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:38.631481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-22d7dba3-d0a5-4275-9c30-52953b5e5ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:38.642947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1102/2184] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 17:51:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:38.665642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac24013-ae3a-4479-ad09-320d1f1c4b8b service swift] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:38.678449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac24013-ae3a-4479-ad09-320d1f1c4b8b service swift] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:38.684040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a1728-0971-4cc5-826f-bf6339c9cffb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:38.685244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a1728-0971-4cc5-826f-bf6339c9cffb service swift] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:38.685938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a1728-0971-4cc5-826f-bf6339c9cffb service swift] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:38.686450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a1728-0971-4cc5-826f-bf6339c9cffb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:38.696139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a1728-0971-4cc5-826f-bf6339c9cffb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:38.698174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a1728-0971-4cc5-826f-bf6339c9cffb service swift] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:38.717474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1083/2185] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:51:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:51:39.005495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7dba3-d0a5-4275-9c30-52953b5e5ae7 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:39.008584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-22d7dba3-d0a5-4275-9c30-52953b5e5ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:39.071984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7dba3-d0a5-4275-9c30-52953b5e5ae7 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:39.080626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:39.126128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:39.126128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:39.126128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] PATH_INFO: `/v3/users/9c4902609ef84c6ca2cbddf03e83df7e` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:39.126128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c4902609ef84c6ca2cbddf03e83df7e)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:39.126128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:39.933720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c4902609ef84c6ca2cbddf03e83df7e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:39.978453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c4902609ef84c6ca2cbddf03e83df7e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:40.183777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c4902609ef84c6ca2cbddf03e83df7e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:40.184423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c4902609ef84c6ca2cbddf03e83df7e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:40.195492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c4902609ef84c6ca2cbddf03e83df7e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:40.207578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1103/2186] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:39 2022] DELETE /identity/v3/users/9c4902609ef84c6ca2cbddf03e83df7e => generated 0 bytes in 1203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:40.211938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d554-b586-44ca-85fd-188a72c4359b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:40.212533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d554-b586-44ca-85fd-188a72c4359b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:40.212533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d554-b586-44ca-85fd-188a72c4359b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:40.220877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:40.250950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-1b57d554-b586-44ca-85fd-188a72c4359b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:40.252258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1b57d554-b586-44ca-85fd-188a72c4359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:40.371190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1084/2187] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:40.536323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:40.538453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:40.578351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5e1b-d523-476c-bc41-76b149e4ebea None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:40.578796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5e1b-d523-476c-bc41-76b149e4ebea None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:40.579095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5e1b-d523-476c-bc41-76b149e4ebea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:40.604947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-22eb5e1b-d523-476c-bc41-76b149e4ebea None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:40.606433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-22eb5e1b-d523-476c-bc41-76b149e4ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:40.675349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f59bd-42c4-44b3-a393-8438ada11159 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:40.686889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:40.687800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:40.688189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:40.688567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:40.702283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1085/2188] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:51:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:40.710156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:40.712687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:40.726436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5e1b-d523-476c-bc41-76b149e4ebea None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:40.732340 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-22eb5e1b-d523-476c-bc41-76b149e4ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:40.732649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1104/2189] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:40.894442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5e1b-d523-476c-bc41-76b149e4ebea None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:40.910927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a84b1-5a08-4e22-80a9-484590ce0d31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:40.911570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a84b1-5a08-4e22-80a9-484590ce0d31 service swift] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:40.911860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a84b1-5a08-4e22-80a9-484590ce0d31 service swift] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:40.913899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a84b1-5a08-4e22-80a9-484590ce0d31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:40.924145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a84b1-5a08-4e22-80a9-484590ce0d31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:40.928036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a84b1-5a08-4e22-80a9-484590ce0d31 service swift] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:40.959184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1086/2190] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:51:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:51:41.833183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:41.834609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:41.863331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc23bd-098f-4997-861e-56920731e81a service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:41.881264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:41.881959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:41.890226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:41.890226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] PATH_INFO: `/v3/projects/bb33707df06144029cb726f5fdab0370` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:41.890226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb33707df06144029cb726f5fdab0370)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:41.890226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:42.149807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:42.189827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb33707df06144029cb726f5fdab0370', 'request_id': 'req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:42.276118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb33707df06144029cb726f5fdab0370', 'request_id': 'req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:42.300269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb33707df06144029cb726f5fdab0370', 'request_id': 'req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:42.320406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] The token cache is being invalidate because project bb33707df06144029cb726f5fdab0370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:42.321350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:42.323461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-657a76c3-07ae-42fa-94c5-06cd52fd3c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:42.332820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1105/2191] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:41 2022] DELETE /identity/v3/projects/bb33707df06144029cb726f5fdab0370 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:42.334895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a84b1-5a08-4e22-80a9-484590ce0d31 service swift] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:42.363395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a84b1-5a08-4e22-80a9-484590ce0d31 service swift] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:42.398063 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348582f-c873-4d28-bbd5-bfd88a3c87dd None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:42.398387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348582f-c873-4d28-bbd5-bfd88a3c87dd None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:42.398730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348582f-c873-4d28-bbd5-bfd88a3c87dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:42.429186 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-8348582f-c873-4d28-bbd5-bfd88a3c87dd None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:42.430979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8348582f-c873-4d28-bbd5-bfd88a3c87dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:42.454430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:42.532182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1106/2192] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 17:51:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:42.565178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348582f-c873-4d28-bbd5-bfd88a3c87dd None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:42.567075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8348582f-c873-4d28-bbd5-bfd88a3c87dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:42.571239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:42.572606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:42.573289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:42.574044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] PATH_INFO: `/v3/users/c7bef72efe334fb89a7be65a8f698b5c` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:42.592596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bef72efe334fb89a7be65a8f698b5c)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:42.597102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:42.779161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348582f-c873-4d28-bbd5-bfd88a3c87dd None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:42.796574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:42.797272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:42.797568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:42.797841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:42.819126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:42.820899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:42.831949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1107/2193] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:51:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:51:42.927156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bef72efe334fb89a7be65a8f698b5c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:43.020028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bef72efe334fb89a7be65a8f698b5c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:43.021627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bef72efe334fb89a7be65a8f698b5c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:43.052861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bef72efe334fb89a7be65a8f698b5c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:43.055197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-29ca491f-97fc-41c7-829e-1335b9e7f5c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bef72efe334fb89a7be65a8f698b5c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:43.142241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1087/2194] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:42 2022] DELETE /identity/v3/users/c7bef72efe334fb89a7be65a8f698b5c => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:44.736271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:44.737948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:44.775689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60ae38-b457-46ec-9c16-015b16621f8d service swift] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:44.962968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:44.989311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:44.992369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:44.993005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:44.993516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] PATH_INFO: `/v3/projects/e3b839c09d4b4f108f3d1c0b5bf5efb2` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:44.995714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b839c09d4b4f108f3d1c0b5bf5efb2)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:45.004602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:45.203246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b839c09d4b4f108f3d1c0b5bf5efb2', 'request_id': 'req-eb558372-2d8b-46da-9d4e-2760a60d30b2'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.358168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b839c09d4b4f108f3d1c0b5bf5efb2', 'request_id': 'req-eb558372-2d8b-46da-9d4e-2760a60d30b2'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.375375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b839c09d4b4f108f3d1c0b5bf5efb2', 'request_id': 'req-eb558372-2d8b-46da-9d4e-2760a60d30b2'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.406241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] The token cache is being invalidate because project e3b839c09d4b4f108f3d1c0b5bf5efb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:45.406951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.409938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb558372-2d8b-46da-9d4e-2760a60d30b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.419661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1108/2195] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:44 2022] DELETE /identity/v3/projects/e3b839c09d4b4f108f3d1c0b5bf5efb2 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:45.424266 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf217f-fca4-4299-81d3-6e651b04c8a1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:45.425024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf217f-fca4-4299-81d3-6e651b04c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:45.425024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf217f-fca4-4299-81d3-6e651b04c8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:45.452756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-51cf217f-fca4-4299-81d3-6e651b04c8a1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:45.456993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-51cf217f-fca4-4299-81d3-6e651b04c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:45.672243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1088/2196] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:45.676753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f62b-2219-4943-9e72-5f1f7f6844c2 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:45.677452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f62b-2219-4943-9e72-5f1f7f6844c2 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:45.678819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f62b-2219-4943-9e72-5f1f7f6844c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:45.711159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-56c8f62b-2219-4943-9e72-5f1f7f6844c2 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:45.712600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-56c8f62b-2219-4943-9e72-5f1f7f6844c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:45.729955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1109/2197] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:45.730854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf217f-fca4-4299-81d3-6e651b04c8a1 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:45.744558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf217f-fca4-4299-81d3-6e651b04c8a1 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:45.768179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:45.768727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:45.768879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:45.769118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:45.770230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:45.770435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:45.772203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:45.800540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1089/2198] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:51:45.805256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8f62b-2219-4943-9e72-5f1f7f6844c2 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:45.818558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8f62b-2219-4943-9e72-5f1f7f6844c2 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:45.829268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6684d-bae6-485d-8361-edf862f2e51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:45.830260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6684d-bae6-485d-8361-edf862f2e51d admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:45.830860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6684d-bae6-485d-8361-edf862f2e51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:45.831288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6684d-bae6-485d-8361-edf862f2e51d admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:45.832904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6684d-bae6-485d-8361-edf862f2e51d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:45.836951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6684d-bae6-485d-8361-edf862f2e51d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:46.250574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1110/2199] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:45 2022] POST /identity/v3/projects => generated 385 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:46.252815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:46.282534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58e827-da5b-47e1-bdfd-129a95349435 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:46.407743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:46.449683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4e47-299e-4e86-a3d0-c9e41a100206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:46.449683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4e47-299e-4e86-a3d0-c9e41a100206 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:46.449683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4e47-299e-4e86-a3d0-c9e41a100206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:46.449683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4e47-299e-4e86-a3d0-c9e41a100206 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:46.451740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d4e47-299e-4e86-a3d0-c9e41a100206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:46.455466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d4e47-299e-4e86-a3d0-c9e41a100206 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:46.623538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1090/2200] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:46 2022] POST /identity/v3/users => generated 293 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:46.627954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6684d-bae6-485d-8361-edf862f2e51d admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:46.645330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6684d-bae6-485d-8361-edf862f2e51d admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:46.662497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dfa99-0ace-42df-b77e-53fe839d1840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:46.663843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dfa99-0ace-42df-b77e-53fe839d1840 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:46.664139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dfa99-0ace-42df-b77e-53fe839d1840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:46.664421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dfa99-0ace-42df-b77e-53fe839d1840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:46.665136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8dfa99-0ace-42df-b77e-53fe839d1840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:46.671952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8dfa99-0ace-42df-b77e-53fe839d1840 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:46.697705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1111/2201] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:46.703518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4e47-299e-4e86-a3d0-c9e41a100206 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:46.729227 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4e47-299e-4e86-a3d0-c9e41a100206 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:46.736268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4ce9a-43a6-47c1-a8cc-dd79902a3a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:46.737278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4ce9a-43a6-47c1-a8cc-dd79902a3a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:46.737669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4ce9a-43a6-47c1-a8cc-dd79902a3a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:46.738095 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4ce9a-43a6-47c1-a8cc-dd79902a3a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:46.738948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4ce9a-43a6-47c1-a8cc-dd79902a3a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:46.750616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4ce9a-43a6-47c1-a8cc-dd79902a3a9d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:46.784548 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1091/2202] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:46.786989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dfa99-0ace-42df-b77e-53fe839d1840 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:46.805042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dfa99-0ace-42df-b77e-53fe839d1840 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:46.817224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885668a1-dfe2-44ed-a714-29f92a4229e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:46.824266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885668a1-dfe2-44ed-a714-29f92a4229e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:46.824266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885668a1-dfe2-44ed-a714-29f92a4229e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:46.824266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885668a1-dfe2-44ed-a714-29f92a4229e4 admin admin] PATH_INFO: `/v3/projects/3238a0d77c25444abd30ba900f464a86/users/d00263ff96cf42fe81141af5d8d0caa6/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:46.866670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885668a1-dfe2-44ed-a714-29f92a4229e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3238a0d77c25444abd30ba900f464a86, user_id=d00263ff96cf42fe81141af5d8d0caa6, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:46.869502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885668a1-dfe2-44ed-a714-29f92a4229e4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:47.027875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1112/2203] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:46 2022] PUT /identity/v3/projects/3238a0d77c25444abd30ba900f464a86/users/d00263ff96cf42fe81141af5d8d0caa6/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:47.041847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec0507-57c0-4b5a-bb9d-48d6ddedd000 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:47.041847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec0507-57c0-4b5a-bb9d-48d6ddedd000 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:47.041847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec0507-57c0-4b5a-bb9d-48d6ddedd000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:47.076971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e0ec0507-57c0-4b5a-bb9d-48d6ddedd000 None None] MFA Rules not processed for user `d00263ff96cf42fe81141af5d8d0caa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:47.078282 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e0ec0507-57c0-4b5a-bb9d-48d6ddedd000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:47.211373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1092/2204] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:47.220324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885668a1-dfe2-44ed-a714-29f92a4229e4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:47.346873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885668a1-dfe2-44ed-a714-29f92a4229e4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:47.352301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:47.353206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:47.353627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:47.354144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:47.354964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:47.358461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:47.603517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1113/2205] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:47 2022] POST /identity/v3/projects => generated 387 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:47.621619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec0507-57c0-4b5a-bb9d-48d6ddedd000 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:47.639889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec0507-57c0-4b5a-bb9d-48d6ddedd000 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:47.642069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:47.647849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:47.664181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:47.734658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ec9ce-c8a4-4e94-8534-7c98f7628d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:47.735754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec9ce-c8a4-4e94-8534-7c98f7628d6e admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:47.736173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec9ce-c8a4-4e94-8534-7c98f7628d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:47.737230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec9ce-c8a4-4e94-8534-7c98f7628d6e admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:47.737498 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ec9ce-c8a4-4e94-8534-7c98f7628d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:47.740369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ec9ce-c8a4-4e94-8534-7c98f7628d6e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:47.744579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3446d6-359f-48ec-b784-2bcd46f4b76d admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:47.776027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:47.777209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:47.778925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:47.779549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] PATH_INFO: `/v3/users/bfdf1fd67f954cde9ad1334bb93ec4e6` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:47.781234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfdf1fd67f954cde9ad1334bb93ec4e6)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:47.784705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:47.920322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1093/2206] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:47 2022] POST /identity/v3/users => generated 300 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:47.923229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ec9ce-c8a4-4e94-8534-7c98f7628d6e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:47.967387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ec9ce-c8a4-4e94-8534-7c98f7628d6e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:47.983902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006af08-f53a-48d4-a3e8-768275308b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:47.984956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006af08-f53a-48d4-a3e8-768275308b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:47.985454 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006af08-f53a-48d4-a3e8-768275308b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:47.986015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006af08-f53a-48d4-a3e8-768275308b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:47.987110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006af08-f53a-48d4-a3e8-768275308b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:47.999938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006af08-f53a-48d4-a3e8-768275308b6b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:48.027584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1094/2207] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:48.033241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006af08-f53a-48d4-a3e8-768275308b6b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:48.039825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfdf1fd67f954cde9ad1334bb93ec4e6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:48.047041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006af08-f53a-48d4-a3e8-768275308b6b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:48.056466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:48.057472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:48.057946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:48.058362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] PATH_INFO: `/v3/projects/8c7daa4f7ef04719a402d1b934eb3dc7/users/d4c9167a719541c5ac2324c760b81ec0/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:48.083507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfdf1fd67f954cde9ad1334bb93ec4e6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:48.095377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7daa4f7ef04719a402d1b934eb3dc7, user_id=d4c9167a719541c5ac2324c760b81ec0, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:48.101831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:48.237825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfdf1fd67f954cde9ad1334bb93ec4e6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:48.239824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfdf1fd67f954cde9ad1334bb93ec4e6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:48.240871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f423cfd1-350b-4200-8fa3-4ea51eb328b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfdf1fd67f954cde9ad1334bb93ec4e6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:48.254334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1095/2208] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:48 2022] PUT /identity/v3/projects/8c7daa4f7ef04719a402d1b934eb3dc7/users/d4c9167a719541c5ac2324c760b81ec0/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:48.263696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:48.268867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:48.276683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1114/2209] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:47 2022] DELETE /identity/v3/users/bfdf1fd67f954cde9ad1334bb93ec4e6 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:48.285071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862c613-0fdc-495a-b9c5-065db5f3df69 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:48.285352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:48.285563 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862c613-0fdc-495a-b9c5-065db5f3df69 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:48.286101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862c613-0fdc-495a-b9c5-065db5f3df69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:48.346651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-9862c613-0fdc-495a-b9c5-065db5f3df69 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:48.355075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9862c613-0fdc-495a-b9c5-065db5f3df69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:48.366423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77d43-9060-4e5c-9148-63641c38c254 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:48.440148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eeafbe-1e31-4a06-942b-1131f318b0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:48.440786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeafbe-1e31-4a06-942b-1131f318b0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:48.441090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeafbe-1e31-4a06-942b-1131f318b0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:48.441489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeafbe-1e31-4a06-942b-1131f318b0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:48.442162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eeafbe-1e31-4a06-942b-1131f318b0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:48.444109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eeafbe-1e31-4a06-942b-1131f318b0e9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:48.447706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1115/2210] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:48.460440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1096/2211] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:48.460968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862c613-0fdc-495a-b9c5-065db5f3df69 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:48.463513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9862c613-0fdc-495a-b9c5-065db5f3df69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:48.464622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eeafbe-1e31-4a06-942b-1131f318b0e9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:48.476159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eeafbe-1e31-4a06-942b-1131f318b0e9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:48.481576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cf918-4049-4d4e-9eb4-6daeb2cc6771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:48.482398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cf918-4049-4d4e-9eb4-6daeb2cc6771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:48.483024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cf918-4049-4d4e-9eb4-6daeb2cc6771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:48.483024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cf918-4049-4d4e-9eb4-6daeb2cc6771 admin admin] PATH_INFO: `/v3/projects/8c7daa4f7ef04719a402d1b934eb3dc7/users/d4c9167a719541c5ac2324c760b81ec0/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:48.486120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cf918-4049-4d4e-9eb4-6daeb2cc6771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7daa4f7ef04719a402d1b934eb3dc7, user_id=d4c9167a719541c5ac2324c760b81ec0, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:48.489022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cf918-4049-4d4e-9eb4-6daeb2cc6771 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:48.501795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1097/2212] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:48 2022] PUT /identity/v3/projects/8c7daa4f7ef04719a402d1b934eb3dc7/users/d4c9167a719541c5ac2324c760b81ec0/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:48.514527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06278729-b5c0-432a-bf5c-d732c5da0801 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:48.514842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06278729-b5c0-432a-bf5c-d732c5da0801 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:48.515148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06278729-b5c0-432a-bf5c-d732c5da0801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:48.552059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-06278729-b5c0-432a-bf5c-d732c5da0801 None None] MFA Rules not processed for user `d4c9167a719541c5ac2324c760b81ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:48.554983 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-06278729-b5c0-432a-bf5c-d732c5da0801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:48.573748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862c613-0fdc-495a-b9c5-065db5f3df69 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:48.664918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:48.670298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d100f2-8bb4-4348-b5cd-03b7cb329d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:48.670802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d100f2-8bb4-4348-b5cd-03b7cb329d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:48.671271 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d100f2-8bb4-4348-b5cd-03b7cb329d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:48.671777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d100f2-8bb4-4348-b5cd-03b7cb329d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:48.694680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1098/2213] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:48.705555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06278729-b5c0-432a-bf5c-d732c5da0801 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:48.707879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-06278729-b5c0-432a-bf5c-d732c5da0801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:48.781505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:48.869776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d100f2-8bb4-4348-b5cd-03b7cb329d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:48.871608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d100f2-8bb4-4348-b5cd-03b7cb329d22 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:48.872780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06278729-b5c0-432a-bf5c-d732c5da0801 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:48.885513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1116/2214] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:48.887028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d9128-eaf9-40f5-af9f-198f76a81fb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:48.888750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9128-eaf9-40f5-af9f-198f76a81fb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:48.889145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9128-eaf9-40f5-af9f-198f76a81fb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:48.889519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9128-eaf9-40f5-af9f-198f76a81fb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:48.916639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d9128-eaf9-40f5-af9f-198f76a81fb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:48.918567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d9128-eaf9-40f5-af9f-198f76a81fb6 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:48.942667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1099/2215] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:48.989005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d100f2-8bb4-4348-b5cd-03b7cb329d22 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:49.001586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d100f2-8bb4-4348-b5cd-03b7cb329d22 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:49.007911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:49.008569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:49.008851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:49.009591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] PATH_INFO: `/v3/users/d00263ff96cf42fe81141af5d8d0caa6` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:49.011062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d00263ff96cf42fe81141af5d8d0caa6)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:49.014334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:49.317701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd00263ff96cf42fe81141af5d8d0caa6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:49.374527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd00263ff96cf42fe81141af5d8d0caa6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:49.506599 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd00263ff96cf42fe81141af5d8d0caa6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:49.508978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd00263ff96cf42fe81141af5d8d0caa6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:49.510586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd00263ff96cf42fe81141af5d8d0caa6'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:49.562253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1117/2216] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:48 2022] DELETE /identity/v3/users/d00263ff96cf42fe81141af5d8d0caa6 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:49.564079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59caa-d63d-4d93-94cd-dc359290c838 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:49.564496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59caa-d63d-4d93-94cd-dc359290c838 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:49.564954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59caa-d63d-4d93-94cd-dc359290c838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:49.615736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-18e59caa-d63d-4d93-94cd-dc359290c838 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:49.618315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-18e59caa-d63d-4d93-94cd-dc359290c838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:49.627142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:49.628648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:49.646524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:49.749608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1100/2217] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:49.772129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59caa-d63d-4d93-94cd-dc359290c838 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:49.772129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-18e59caa-d63d-4d93-94cd-dc359290c838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:49.781240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf96de7-e371-44f1-8eb2-6ba5718f7150 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:49.792105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:49.793811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:49.794757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:49.795775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] PATH_INFO: `/v3/projects/a770f0caef364853a4033422b76ab918` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:49.797712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] RBAC: Authorizing `identity:delete_project(project_id=a770f0caef364853a4033422b76ab918)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:49.800927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:49.874443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59caa-d63d-4d93-94cd-dc359290c838 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:49.880531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da375035-5cd3-4f95-b6b1-2afbd95a28ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:49.882204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da375035-5cd3-4f95-b6b1-2afbd95a28ba service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:49.883149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da375035-5cd3-4f95-b6b1-2afbd95a28ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:49.884268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da375035-5cd3-4f95-b6b1-2afbd95a28ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:49.903155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da375035-5cd3-4f95-b6b1-2afbd95a28ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:49.905226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da375035-5cd3-4f95-b6b1-2afbd95a28ba service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:49.947423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1101/2218] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:49.995548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a770f0caef364853a4033422b76ab918', 'request_id': 'req-96d32b8c-0c92-47de-a870-12a98588a1af'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:50.075831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a770f0caef364853a4033422b76ab918', 'request_id': 'req-96d32b8c-0c92-47de-a870-12a98588a1af'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:50.089048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a770f0caef364853a4033422b76ab918', 'request_id': 'req-96d32b8c-0c92-47de-a870-12a98588a1af'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:50.113780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] The token cache is being invalidate because project a770f0caef364853a4033422b76ab918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:50.114392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:50.115381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-96d32b8c-0c92-47de-a870-12a98588a1af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:50.117924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1118/2219] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:49 2022] DELETE /identity/v3/projects/a770f0caef364853a4033422b76ab918 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:50.125325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b0fd-c2e1-488c-9e9c-05da446cd6b1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:50.126524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b0fd-c2e1-488c-9e9c-05da446cd6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:50.126524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b0fd-c2e1-488c-9e9c-05da446cd6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:50.149116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-3a73b0fd-c2e1-488c-9e9c-05da446cd6b1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:50.150623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3a73b0fd-c2e1-488c-9e9c-05da446cd6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:50.327858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1102/2220] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:50.345137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26569971-7635-4949-b5f2-baf0e1737ed9 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:50.347538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26569971-7635-4949-b5f2-baf0e1737ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:50.348530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26569971-7635-4949-b5f2-baf0e1737ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:50.431056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-26569971-7635-4949-b5f2-baf0e1737ed9 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:50.433105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-26569971-7635-4949-b5f2-baf0e1737ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:50.454434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1119/2221] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:50.457822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b0fd-c2e1-488c-9e9c-05da446cd6b1 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:50.473719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b0fd-c2e1-488c-9e9c-05da446cd6b1 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:50.480400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:50.481687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:50.485108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:50.485108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:50.485108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:50.485108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:50.485811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:50.489957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:50.505586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1103/2222] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:50.506961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26569971-7635-4949-b5f2-baf0e1737ed9 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:50.519715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26569971-7635-4949-b5f2-baf0e1737ed9 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:50.529670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:50.530864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:50.531340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:50.531801 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:50.532647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:50.535718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:50.754555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1120/2223] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:50 2022] POST /identity/v3/projects => generated 379 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:50.755123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:50.773833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130233e4-16ee-4f72-bab9-876c591490af admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:50.849525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:50.851528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:50.938432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537484e0-51fd-4822-9d14-7e62c8a26214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:50.939665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537484e0-51fd-4822-9d14-7e62c8a26214 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:50.939665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537484e0-51fd-4822-9d14-7e62c8a26214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:50.941765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537484e0-51fd-4822-9d14-7e62c8a26214 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:50.941765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537484e0-51fd-4822-9d14-7e62c8a26214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:50.943192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537484e0-51fd-4822-9d14-7e62c8a26214 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:50.994578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1880e-1411-47d4-b5b0-3022010e5892 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:51.001637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:51.002618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:51.003004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:51.003387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] PATH_INFO: `/v3/projects/3238a0d77c25444abd30ba900f464a86` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:51.004795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3238a0d77c25444abd30ba900f464a86)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:51.006839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:51.042841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:51.099403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1104/2224] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:50 2022] POST /identity/v3/users => generated 290 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:51.103815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537484e0-51fd-4822-9d14-7e62c8a26214 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:51.114817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537484e0-51fd-4822-9d14-7e62c8a26214 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:51.119831 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896353c-2595-4f85-a3b7-87d9a43752d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:51.120516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896353c-2595-4f85-a3b7-87d9a43752d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:51.120740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896353c-2595-4f85-a3b7-87d9a43752d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:51.121013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896353c-2595-4f85-a3b7-87d9a43752d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:51.122395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896353c-2595-4f85-a3b7-87d9a43752d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:51.124179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896353c-2595-4f85-a3b7-87d9a43752d6 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:51.142141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1105/2225] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:51.143157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896353c-2595-4f85-a3b7-87d9a43752d6 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:51.155218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896353c-2595-4f85-a3b7-87d9a43752d6 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:51.171617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b45eb-e856-4b2c-99bf-54c3d53053b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:51.175049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b45eb-e856-4b2c-99bf-54c3d53053b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:51.176288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b45eb-e856-4b2c-99bf-54c3d53053b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:51.177252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b45eb-e856-4b2c-99bf-54c3d53053b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:51.178889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b45eb-e856-4b2c-99bf-54c3d53053b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:51.185877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b45eb-e856-4b2c-99bf-54c3d53053b8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:51.203638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1106/2226] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:51.209615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3238a0d77c25444abd30ba900f464a86', 'request_id': 'req-16753b81-fae4-4b1d-b40e-c317cb9b1db3'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.219728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b45eb-e856-4b2c-99bf-54c3d53053b8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:51.230540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b45eb-e856-4b2c-99bf-54c3d53053b8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:51.295633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e10ba-bcd6-4991-a6fb-f83173a2bf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:51.296211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e10ba-bcd6-4991-a6fb-f83173a2bf2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:51.296494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e10ba-bcd6-4991-a6fb-f83173a2bf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:51.296769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e10ba-bcd6-4991-a6fb-f83173a2bf2b admin admin] PATH_INFO: `/v3/projects/54cebab3bf1f440dbaa4ef020177a4bb/users/f19bf0e6a8ad425cbbf804d6d7be5a0a/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:51.313786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e10ba-bcd6-4991-a6fb-f83173a2bf2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=54cebab3bf1f440dbaa4ef020177a4bb, user_id=f19bf0e6a8ad425cbbf804d6d7be5a0a, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:51.316570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e10ba-bcd6-4991-a6fb-f83173a2bf2b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:51.349176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3238a0d77c25444abd30ba900f464a86', 'request_id': 'req-16753b81-fae4-4b1d-b40e-c317cb9b1db3'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.357130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3238a0d77c25444abd30ba900f464a86', 'request_id': 'req-16753b81-fae4-4b1d-b40e-c317cb9b1db3'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.369096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] The token cache is being invalidate because project 3238a0d77c25444abd30ba900f464a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:51.369390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.370309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.372408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1121/2227] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:50 2022] DELETE /identity/v3/projects/3238a0d77c25444abd30ba900f464a86 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:51.376684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:51.378173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:51.392676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753b81-fae4-4b1d-b40e-c317cb9b1db3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:51.397978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:51.398647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:51.398923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:51.399196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] PATH_INFO: `/v3/users/d4c9167a719541c5ac2324c760b81ec0` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:51.400379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c9167a719541c5ac2324c760b81ec0)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:51.402640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:51.424637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1107/2228] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:51 2022] PUT /identity/v3/projects/54cebab3bf1f440dbaa4ef020177a4bb/users/f19bf0e6a8ad425cbbf804d6d7be5a0a/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:51.429981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919271a-03cd-4d5c-afc7-56e51efb9e70 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:51.430288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919271a-03cd-4d5c-afc7-56e51efb9e70 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:51.430544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919271a-03cd-4d5c-afc7-56e51efb9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:51.451201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a919271a-03cd-4d5c-afc7-56e51efb9e70 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:51.452471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a919271a-03cd-4d5c-afc7-56e51efb9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:51.500108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:51.521462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1108/2229] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:51.534784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919271a-03cd-4d5c-afc7-56e51efb9e70 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:51.535804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a919271a-03cd-4d5c-afc7-56e51efb9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:51.606512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919271a-03cd-4d5c-afc7-56e51efb9e70 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:51.625071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d315d65-e1e2-4dae-a53f-32d115059c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:51.625722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315d65-e1e2-4dae-a53f-32d115059c00 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:51.626235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315d65-e1e2-4dae-a53f-32d115059c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:51.626364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315d65-e1e2-4dae-a53f-32d115059c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:51.638197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d315d65-e1e2-4dae-a53f-32d115059c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:51.639927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d315d65-e1e2-4dae-a53f-32d115059c00 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:51.652680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1109/2230] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:51.662846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c9167a719541c5ac2324c760b81ec0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.685514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c9167a719541c5ac2324c760b81ec0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.772544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c9167a719541c5ac2324c760b81ec0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.773273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c9167a719541c5ac2324c760b81ec0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.774012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-eb969318-a68c-4ce3-8d12-8d785f2fb23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c9167a719541c5ac2324c760b81ec0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:51.810974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1122/2231] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:51 2022] DELETE /identity/v3/users/d4c9167a719541c5ac2324c760b81ec0 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:52.651947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d315d65-e1e2-4dae-a53f-32d115059c00 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:52.673452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d315d65-e1e2-4dae-a53f-32d115059c00 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:52.784252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:52.784920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:52.785194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:52.785464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] PATH_INFO: `/v3/projects/8c7daa4f7ef04719a402d1b934eb3dc7` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:52.787223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7daa4f7ef04719a402d1b934eb3dc7)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:52.789699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:52.927342 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7daa4f7ef04719a402d1b934eb3dc7', 'request_id': 'req-4ec9c600-14e6-4dba-b683-436975bc2f9a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:53.050814 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7daa4f7ef04719a402d1b934eb3dc7', 'request_id': 'req-4ec9c600-14e6-4dba-b683-436975bc2f9a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:53.065298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7daa4f7ef04719a402d1b934eb3dc7', 'request_id': 'req-4ec9c600-14e6-4dba-b683-436975bc2f9a'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:53.091947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] The token cache is being invalidate because project 8c7daa4f7ef04719a402d1b934eb3dc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:53.092262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:53.092988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4ec9c600-14e6-4dba-b683-436975bc2f9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:53.095777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1110/2232] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:52 2022] DELETE /identity/v3/projects/8c7daa4f7ef04719a402d1b934eb3dc7 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:53.100009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd19397-470d-45df-b9c1-b3eb5b902a31 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:53.100864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd19397-470d-45df-b9c1-b3eb5b902a31 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:53.101303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd19397-470d-45df-b9c1-b3eb5b902a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:53.147342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3dd19397-470d-45df-b9c1-b3eb5b902a31 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:53.148619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3dd19397-470d-45df-b9c1-b3eb5b902a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:53.248712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1123/2233] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:53.257894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d12520-1baa-4311-b65c-b7447183d438 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:53.258429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d12520-1baa-4311-b65c-b7447183d438 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:53.259144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d12520-1baa-4311-b65c-b7447183d438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:53.286643 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-13d12520-1baa-4311-b65c-b7447183d438 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:53.287587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-13d12520-1baa-4311-b65c-b7447183d438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:53.298014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1111/2234] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:53.303226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd19397-470d-45df-b9c1-b3eb5b902a31 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:53.322663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd19397-470d-45df-b9c1-b3eb5b902a31 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:53.330932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:53.331978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:53.332459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:53.332957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:53.334325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:53.334826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:53.338073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:53.347974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:53.359657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1124/2235] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:53.361750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d12520-1baa-4311-b65c-b7447183d438 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:53.374281 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d12520-1baa-4311-b65c-b7447183d438 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:53.381950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782be414-702b-49f3-8121-60a978d86d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:53.382664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782be414-702b-49f3-8121-60a978d86d4b admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:53.382821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782be414-702b-49f3-8121-60a978d86d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:53.383123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782be414-702b-49f3-8121-60a978d86d4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:53.383751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782be414-702b-49f3-8121-60a978d86d4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:53.385679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782be414-702b-49f3-8121-60a978d86d4b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:53.401359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:53.545131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1112/2236] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:53 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:53.549900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:53.561810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cafd94-d583-4ab6-b745-3d0482b7b418 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:53.625881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ae04a-d4fa-4768-ae8b-b131552a2e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:53.626507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ae04a-d4fa-4768-ae8b-b131552a2e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:53.626821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ae04a-d4fa-4768-ae8b-b131552a2e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:53.627130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ae04a-d4fa-4768-ae8b-b131552a2e58 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:53.627764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ae04a-d4fa-4768-ae8b-b131552a2e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:53.629697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ae04a-d4fa-4768-ae8b-b131552a2e58 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:53.889314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1125/2237] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:53 2022] POST /identity/v3/users => generated 294 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:53.894725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782be414-702b-49f3-8121-60a978d86d4b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:53.908793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782be414-702b-49f3-8121-60a978d86d4b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:53.916866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5906be0-5910-4783-a772-17ecdd6d9cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:53.917571 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5906be0-5910-4783-a772-17ecdd6d9cad admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:53.917880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5906be0-5910-4783-a772-17ecdd6d9cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:53.918157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5906be0-5910-4783-a772-17ecdd6d9cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:53.919784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5906be0-5910-4783-a772-17ecdd6d9cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:53.924522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5906be0-5910-4783-a772-17ecdd6d9cad admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:53.938446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1113/2238] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:53.943536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ae04a-d4fa-4768-ae8b-b131552a2e58 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:53.965981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ae04a-d4fa-4768-ae8b-b131552a2e58 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:53.974158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c48244-a196-4909-ae4b-af20a6095054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:53.974798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c48244-a196-4909-ae4b-af20a6095054 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:53.975080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c48244-a196-4909-ae4b-af20a6095054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:53.975349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c48244-a196-4909-ae4b-af20a6095054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:53.976086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c48244-a196-4909-ae4b-af20a6095054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:53.979019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c48244-a196-4909-ae4b-af20a6095054 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:53.996774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1126/2239] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:53.999513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5906be0-5910-4783-a772-17ecdd6d9cad admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:54.027483 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5906be0-5910-4783-a772-17ecdd6d9cad admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:54.040167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089437f8-4d84-464f-b5ac-dfe591d7fdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:54.042536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089437f8-4d84-464f-b5ac-dfe591d7fdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:54.043203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089437f8-4d84-464f-b5ac-dfe591d7fdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:54.044609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089437f8-4d84-464f-b5ac-dfe591d7fdc7 admin admin] PATH_INFO: `/v3/projects/3f04161193df44febc6e8b3fa93621dd/users/7acbc2ab678f432b9f2569ee7f5adc56/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:54.076306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089437f8-4d84-464f-b5ac-dfe591d7fdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f04161193df44febc6e8b3fa93621dd, user_id=7acbc2ab678f432b9f2569ee7f5adc56, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:54.079121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089437f8-4d84-464f-b5ac-dfe591d7fdc7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:54.225341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1114/2240] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:53 2022] PUT /identity/v3/projects/3f04161193df44febc6e8b3fa93621dd/users/7acbc2ab678f432b9f2569ee7f5adc56/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:54.245786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f4cd4-85a4-4e72-8c2f-28196f7bff33 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:54.246724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f4cd4-85a4-4e72-8c2f-28196f7bff33 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:54.246865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f4cd4-85a4-4e72-8c2f-28196f7bff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:54.289247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-097f4cd4-85a4-4e72-8c2f-28196f7bff33 None None] MFA Rules not processed for user `7acbc2ab678f432b9f2569ee7f5adc56`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:54.290721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-097f4cd4-85a4-4e72-8c2f-28196f7bff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:54.408440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1127/2241] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:54.409915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089437f8-4d84-464f-b5ac-dfe591d7fdc7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:54.503014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089437f8-4d84-464f-b5ac-dfe591d7fdc7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:54.509038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abed6-adc7-4b90-8053-779600e4ddf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:54.509892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abed6-adc7-4b90-8053-779600e4ddf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:54.510293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abed6-adc7-4b90-8053-779600e4ddf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:54.510698 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abed6-adc7-4b90-8053-779600e4ddf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:54.511537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866abed6-adc7-4b90-8053-779600e4ddf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:54.514485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866abed6-adc7-4b90-8053-779600e4ddf7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:54.694742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1115/2242] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:54 2022] POST /identity/v3/projects => generated 385 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:54.699117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f4cd4-85a4-4e72-8c2f-28196f7bff33 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:54.713724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f4cd4-85a4-4e72-8c2f-28196f7bff33 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:54.816165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4d2c7-8447-43cc-b9b0-755f162a34c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:54.817028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4d2c7-8447-43cc-b9b0-755f162a34c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:54.817211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4d2c7-8447-43cc-b9b0-755f162a34c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:54.817423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4d2c7-8447-43cc-b9b0-755f162a34c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:54.818419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4d2c7-8447-43cc-b9b0-755f162a34c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:54.823683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4d2c7-8447-43cc-b9b0-755f162a34c3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:54.939633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1128/2243] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:54 2022] POST /identity/v3/users => generated 299 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:54.941655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abed6-adc7-4b90-8053-779600e4ddf7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:54.960971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abed6-adc7-4b90-8053-779600e4ddf7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:54.968247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7fe76-f61e-4316-98be-4b93fce8545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:54.969029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7fe76-f61e-4316-98be-4b93fce8545e admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:54.969461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7fe76-f61e-4316-98be-4b93fce8545e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:54.970003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7fe76-f61e-4316-98be-4b93fce8545e admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:54.971209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7fe76-f61e-4316-98be-4b93fce8545e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:54.974278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7fe76-f61e-4316-98be-4b93fce8545e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:54.998621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1116/2244] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:55.000461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4d2c7-8447-43cc-b9b0-755f162a34c3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:55.011998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4d2c7-8447-43cc-b9b0-755f162a34c3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:55.019507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40b0e5-9643-4415-a51b-584aa1505927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:55.020634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40b0e5-9643-4415-a51b-584aa1505927 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:55.020978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40b0e5-9643-4415-a51b-584aa1505927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:55.021247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40b0e5-9643-4415-a51b-584aa1505927 admin admin] PATH_INFO: `/v3/projects/427acd07fea14506a19ba053a842474a/users/6235576537fc490db0f1c96a00117c2f/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:55.047608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40b0e5-9643-4415-a51b-584aa1505927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427acd07fea14506a19ba053a842474a, user_id=6235576537fc490db0f1c96a00117c2f, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:55.054414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40b0e5-9643-4415-a51b-584aa1505927 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:55.180828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1129/2245] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:54 2022] PUT /identity/v3/projects/427acd07fea14506a19ba053a842474a/users/6235576537fc490db0f1c96a00117c2f/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:55.187669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7fe76-f61e-4316-98be-4b93fce8545e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:55.200689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7fe76-f61e-4316-98be-4b93fce8545e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:55.287072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08a9f8-a824-42a1-a05f-c696d721fa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:55.287072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08a9f8-a824-42a1-a05f-c696d721fa3b admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:55.287072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08a9f8-a824-42a1-a05f-c696d721fa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:55.291631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08a9f8-a824-42a1-a05f-c696d721fa3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:55.291631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08a9f8-a824-42a1-a05f-c696d721fa3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:55.296132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08a9f8-a824-42a1-a05f-c696d721fa3b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:55.314742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1117/2246] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:55.318572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40b0e5-9643-4415-a51b-584aa1505927 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:55.333556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40b0e5-9643-4415-a51b-584aa1505927 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:55.350553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:55.350553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:55.350553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:55.350894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] PATH_INFO: `/v3/projects/427acd07fea14506a19ba053a842474a/users/6235576537fc490db0f1c96a00117c2f/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:55.353696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427acd07fea14506a19ba053a842474a, user_id=6235576537fc490db0f1c96a00117c2f, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:55.356533 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:55.377838 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1130/2247] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:55 2022] PUT /identity/v3/projects/427acd07fea14506a19ba053a842474a/users/6235576537fc490db0f1c96a00117c2f/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:55.400534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21616ae5-17a9-4268-94aa-ef7b3c851c91 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:55.400804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21616ae5-17a9-4268-94aa-ef7b3c851c91 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:55.401352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21616ae5-17a9-4268-94aa-ef7b3c851c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:55.458914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-21616ae5-17a9-4268-94aa-ef7b3c851c91 None None] MFA Rules not processed for user `6235576537fc490db0f1c96a00117c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:55.460214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-21616ae5-17a9-4268-94aa-ef7b3c851c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:55.563953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:55.598565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1118/2248] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:55.611872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:55.614003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:55.689877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b953d3-f179-420c-bbe0-bcad59e41e42 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:55.701175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a84f2-10fd-4e1d-b13d-ee4517d8f10f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:55.702527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a84f2-10fd-4e1d-b13d-ee4517d8f10f service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:55.702805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a84f2-10fd-4e1d-b13d-ee4517d8f10f service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:55.703088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a84f2-10fd-4e1d-b13d-ee4517d8f10f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:55.715670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a84f2-10fd-4e1d-b13d-ee4517d8f10f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:55.721267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a84f2-10fd-4e1d-b13d-ee4517d8f10f service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:55.729324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1131/2249] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:55.818347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:55.830820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21616ae5-17a9-4268-94aa-ef7b3c851c91 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:56.007058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21616ae5-17a9-4268-94aa-ef7b3c851c91 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:56.019280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:56.020478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:56.020944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:56.021272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] PATH_INFO: `/v3/users/7acbc2ab678f432b9f2569ee7f5adc56` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:56.023619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7acbc2ab678f432b9f2569ee7f5adc56)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:56.028717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:56.271306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7acbc2ab678f432b9f2569ee7f5adc56'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:56.407447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7acbc2ab678f432b9f2569ee7f5adc56'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:56.408810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7acbc2ab678f432b9f2569ee7f5adc56'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:56.437645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7acbc2ab678f432b9f2569ee7f5adc56'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:56.438646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7acbc2ab678f432b9f2569ee7f5adc56'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:56.493708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1119/2250] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:55 2022] DELETE /identity/v3/users/7acbc2ab678f432b9f2569ee7f5adc56 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:56.498740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbbec-0844-44f8-95d1-db0b6347189d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:56.502274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbbec-0844-44f8-95d1-db0b6347189d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:56.503334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbbec-0844-44f8-95d1-db0b6347189d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:56.540484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-517dbbec-0844-44f8-95d1-db0b6347189d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:56.542248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-517dbbec-0844-44f8-95d1-db0b6347189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:56.637120 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1132/2251] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:56.669390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:56.671110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:56.814585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef94e00-9a56-4a1f-b048-1a325d0fabdf admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:56.819779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34f8a7-b133-4ccf-b1e3-e683e4970f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:56.820578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34f8a7-b133-4ccf-b1e3-e683e4970f3b service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:56.820905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34f8a7-b133-4ccf-b1e3-e683e4970f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:56.821392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34f8a7-b133-4ccf-b1e3-e683e4970f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:56.834940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34f8a7-b133-4ccf-b1e3-e683e4970f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:56.837178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34f8a7-b133-4ccf-b1e3-e683e4970f3b service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:56.855083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1120/2252] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:56.898567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:57.446948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbbec-0844-44f8-95d1-db0b6347189d None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:57.446948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-517dbbec-0844-44f8-95d1-db0b6347189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:57.480431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbbec-0844-44f8-95d1-db0b6347189d None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:57.505512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:57.508616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:57.508616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:57.508616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] PATH_INFO: `/v3/projects/3f04161193df44febc6e8b3fa93621dd` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:57.509256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f04161193df44febc6e8b3fa93621dd)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:57.515724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:57.782477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f04161193df44febc6e8b3fa93621dd', 'request_id': 'req-aa152dbc-6068-4967-8bd1-783a0d8a88dd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:57.885603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f04161193df44febc6e8b3fa93621dd', 'request_id': 'req-aa152dbc-6068-4967-8bd1-783a0d8a88dd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:57.888415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:57.901555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f04161193df44febc6e8b3fa93621dd', 'request_id': 'req-aa152dbc-6068-4967-8bd1-783a0d8a88dd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:57.920233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] The token cache is being invalidate because project 3f04161193df44febc6e8b3fa93621dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:57.920651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:57.922037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-aa152dbc-6068-4967-8bd1-783a0d8a88dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:57.923749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1133/2253] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:57 2022] DELETE /identity/v3/projects/3f04161193df44febc6e8b3fa93621dd => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:57.928602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34f8a7-b133-4ccf-b1e3-e683e4970f3b service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:57.943818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34f8a7-b133-4ccf-b1e3-e683e4970f3b service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:58.086555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:58.087541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:58.088001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:58.088493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] PATH_INFO: `/v3/users/6235576537fc490db0f1c96a00117c2f` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:58.089924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6235576537fc490db0f1c96a00117c2f)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:58.093328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:58.459197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6235576537fc490db0f1c96a00117c2f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:58.583423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6235576537fc490db0f1c96a00117c2f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:58.584217 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6235576537fc490db0f1c96a00117c2f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:58.594887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6235576537fc490db0f1c96a00117c2f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:58.595693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2745908a-452c-470d-b96b-16d094aaa149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6235576537fc490db0f1c96a00117c2f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:58.625427 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1121/2254] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:57 2022] DELETE /identity/v3/users/6235576537fc490db0f1c96a00117c2f => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:58.882680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c368c54-ee48-44bd-8b80-41d82bd88d0a None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:58.883260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c368c54-ee48-44bd-8b80-41d82bd88d0a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:58.883673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c368c54-ee48-44bd-8b80-41d82bd88d0a None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:58.885181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1134/2255] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:51:58.893258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159b0ae-8ded-40f7-8ee9-d6a6db4936f6 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:58.893882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159b0ae-8ded-40f7-8ee9-d6a6db4936f6 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:58.894804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159b0ae-8ded-40f7-8ee9-d6a6db4936f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:58.925317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9159b0ae-8ded-40f7-8ee9-d6a6db4936f6 None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:58.927189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9159b0ae-8ded-40f7-8ee9-d6a6db4936f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:59.012487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:59.072598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1122/2256] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:51:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:59.091717 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c368c54-ee48-44bd-8b80-41d82bd88d0a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:59.094201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4c368c54-ee48-44bd-8b80-41d82bd88d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:59.262944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c368c54-ee48-44bd-8b80-41d82bd88d0a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:59.262944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd416e-5aa6-44fe-9653-a8bd8860ea8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d874536170494496769b4f37234b2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:59.262944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd416e-5aa6-44fe-9653-a8bd8860ea8e service glance] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:59.262944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd416e-5aa6-44fe-9653-a8bd8860ea8e service glance] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:59.263331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd416e-5aa6-44fe-9653-a8bd8860ea8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:59.273933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd416e-5aa6-44fe-9653-a8bd8860ea8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:59.276043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd416e-5aa6-44fe-9653-a8bd8860ea8e service glance] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:59.300230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1135/2257] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:59.415839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159b0ae-8ded-40f7-8ee9-d6a6db4936f6 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:59.417756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9159b0ae-8ded-40f7-8ee9-d6a6db4936f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:59.562509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0304-a883-4cb9-aa08-c64ffc322386 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:59.563017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0304-a883-4cb9-aa08-c64ffc322386 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:59.563491 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0304-a883-4cb9-aa08-c64ffc322386 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:59.563912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159b0ae-8ded-40f7-8ee9-d6a6db4936f6 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:59.565244 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1136/2258] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:51:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:51:59.575530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:59.575530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:59.575530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:59.575530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] PATH_INFO: `/v3/projects/427acd07fea14506a19ba053a842474a` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:59.576277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=427acd07fea14506a19ba053a842474a)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:59.577919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34e365-0916-42be-adb1-6c50fc8334cc None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:59.578424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34e365-0916-42be-adb1-6c50fc8334cc None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:59.578787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:59.579040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34e365-0916-42be-adb1-6c50fc8334cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:59.615645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-9b34e365-0916-42be-adb1-6c50fc8334cc None None] MFA Rules not processed for user `efc068011c14492d9ba483650cf632e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:59.617479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9b34e365-0916-42be-adb1-6c50fc8334cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:59.725670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1137/2259] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:51:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:59.739514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34e365-0916-42be-adb1-6c50fc8334cc None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:59.740959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9b34e365-0916-42be-adb1-6c50fc8334cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:59.753706 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427acd07fea14506a19ba053a842474a', 'request_id': 'req-c15da516-8189-49b7-b2f3-27e80f40e866'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:59.756530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34e365-0916-42be-adb1-6c50fc8334cc None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:59.812682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427acd07fea14506a19ba053a842474a', 'request_id': 'req-c15da516-8189-49b7-b2f3-27e80f40e866'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:59.821386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427acd07fea14506a19ba053a842474a', 'request_id': 'req-c15da516-8189-49b7-b2f3-27e80f40e866'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:59.829527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451cca1-d37d-4282-a371-a10387a20944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:59.830450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451cca1-d37d-4282-a371-a10387a20944 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:59.831056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451cca1-d37d-4282-a371-a10387a20944 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:59.831350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451cca1-d37d-4282-a371-a10387a20944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:59.836979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] The token cache is being invalidate because project 427acd07fea14506a19ba053a842474a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:59.841037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:59.841037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c15da516-8189-49b7-b2f3-27e80f40e866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:59.848934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451cca1-d37d-4282-a371-a10387a20944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:59.851108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451cca1-d37d-4282-a371-a10387a20944 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:59.851956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1123/2260] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:59 2022] DELETE /identity/v3/projects/427acd07fea14506a19ba053a842474a => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:59.853478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e307d0-b91a-4441-aa73-449041923da3 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:59.853769 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e307d0-b91a-4441-aa73-449041923da3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:59.854096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e307d0-b91a-4441-aa73-449041923da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:59.864474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1138/2261] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:59.890292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-70e307d0-b91a-4441-aa73-449041923da3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:59.892158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-70e307d0-b91a-4441-aa73-449041923da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:00.000292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4bba-c653-4aba-827c-f6e268157ae3 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.001081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4bba-c653-4aba-827c-f6e268157ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.001081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4bba-c653-4aba-827c-f6e268157ae3 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.003325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1139/2262] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 17:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:52:00.011384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60420d-4f04-426d-9c73-bd19c8b0fd3b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.012363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60420d-4f04-426d-9c73-bd19c8b0fd3b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.012831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60420d-4f04-426d-9c73-bd19c8b0fd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.044533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1124/2263] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:00.053631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4c70e-94be-49b6-9c23-d3d6fa844273 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.053941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4c70e-94be-49b6-9c23-d3d6fa844273 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.054229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4c70e-94be-49b6-9c23-d3d6fa844273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.070452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-0f60420d-4f04-426d-9c73-bd19c8b0fd3b None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:00.072904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0f60420d-4f04-426d-9c73-bd19c8b0fd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:00.081757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-87a4c70e-94be-49b6-9c23-d3d6fa844273 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:00.084017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-87a4c70e-94be-49b6-9c23-d3d6fa844273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:00.101905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1125/2264] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:52:00.107802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4c70e-94be-49b6-9c23-d3d6fa844273 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.125936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4c70e-94be-49b6-9c23-d3d6fa844273 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.142094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:00.144394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.145027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.152059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.160584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.160584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:52:00.160584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.160584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:00.171685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:00.184311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1126/2265] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:52:00.188376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.214349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65069824-656d-4c46-81ca-ffad69d30f26 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.231961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1140/2266] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 17:52:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:00.246885 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60420d-4f04-426d-9c73-bd19c8b0fd3b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.249924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0f60420d-4f04-426d-9c73-bd19c8b0fd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:00.262565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7223aba-6eab-4d5c-8da6-a4b58f571163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.265317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7223aba-6eab-4d5c-8da6-a4b58f571163 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.268724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7223aba-6eab-4d5c-8da6-a4b58f571163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.272876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7223aba-6eab-4d5c-8da6-a4b58f571163 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.274186 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7223aba-6eab-4d5c-8da6-a4b58f571163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.286823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60420d-4f04-426d-9c73-bd19c8b0fd3b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.287133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7223aba-6eab-4d5c-8da6-a4b58f571163 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:00.291667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3382dbc-eb17-4bc7-8f1b-735645d46ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.291971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382dbc-eb17-4bc7-8f1b-735645d46ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.292402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382dbc-eb17-4bc7-8f1b-735645d46ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.292745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382dbc-eb17-4bc7-8f1b-735645d46ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.318935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382dbc-eb17-4bc7-8f1b-735645d46ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.321124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382dbc-eb17-4bc7-8f1b-735645d46ae4 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:00.360210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1141/2267] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:52:00.428024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1127/2268] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:00 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:52:00.432833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3382dbc-eb17-4bc7-8f1b-735645d46ae4 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.448468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3382dbc-eb17-4bc7-8f1b-735645d46ae4 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.571249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01450a6-0d2e-48bb-b6b1-6044ac7e3501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.571974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01450a6-0d2e-48bb-b6b1-6044ac7e3501 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.572399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01450a6-0d2e-48bb-b6b1-6044ac7e3501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.572779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01450a6-0d2e-48bb-b6b1-6044ac7e3501 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.573684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01450a6-0d2e-48bb-b6b1-6044ac7e3501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.578845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01450a6-0d2e-48bb-b6b1-6044ac7e3501 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:00.580617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71269d-3f47-4934-bb72-2c2471fe01cf None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.581338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71269d-3f47-4934-bb72-2c2471fe01cf None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.581717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71269d-3f47-4934-bb72-2c2471fe01cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.652592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-1b71269d-3f47-4934-bb72-2c2471fe01cf None None] MFA Rules not processed for user `f19bf0e6a8ad425cbbf804d6d7be5a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:00.654706 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1b71269d-3f47-4934-bb72-2c2471fe01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:00.701575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1142/2269] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:00 2022] POST /identity/v3/users => generated 292 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:00.702870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01450a6-0d2e-48bb-b6b1-6044ac7e3501 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.725376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01450a6-0d2e-48bb-b6b1-6044ac7e3501 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.744124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20171a8f-3335-47cc-bb43-b474f7dedcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.744829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20171a8f-3335-47cc-bb43-b474f7dedcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.745117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20171a8f-3335-47cc-bb43-b474f7dedcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.747569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20171a8f-3335-47cc-bb43-b474f7dedcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.748301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20171a8f-3335-47cc-bb43-b474f7dedcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.750577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20171a8f-3335-47cc-bb43-b474f7dedcf9 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:00.779383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1143/2270] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:00.785633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20171a8f-3335-47cc-bb43-b474f7dedcf9 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.800448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1128/2271] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:00.808891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71269d-3f47-4934-bb72-2c2471fe01cf None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.810899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1b71269d-3f47-4934-bb72-2c2471fe01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:00.822918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20171a8f-3335-47cc-bb43-b474f7dedcf9 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.827285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71269d-3f47-4934-bb72-2c2471fe01cf None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.828983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbfd0e-0589-4164-8981-474b2031df79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.829695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbfd0e-0589-4164-8981-474b2031df79 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.830025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbfd0e-0589-4164-8981-474b2031df79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.830306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbfd0e-0589-4164-8981-474b2031df79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.831324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbfd0e-0589-4164-8981-474b2031df79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.832960 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbfd0e-0589-4164-8981-474b2031df79 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:00.833338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59ee94-c6e1-4fdd-b5a1-4c3b94862147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.834072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ee94-c6e1-4fdd-b5a1-4c3b94862147 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.834423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ee94-c6e1-4fdd-b5a1-4c3b94862147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.834765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ee94-c6e1-4fdd-b5a1-4c3b94862147 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.835494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59ee94-c6e1-4fdd-b5a1-4c3b94862147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.839403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59ee94-c6e1-4fdd-b5a1-4c3b94862147 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:00.846651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1144/2272] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:00.858848 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbfd0e-0589-4164-8981-474b2031df79 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.875689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbfd0e-0589-4164-8981-474b2031df79 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.879399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e7791-2099-4bb7-ac5d-7df97750291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.880091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e7791-2099-4bb7-ac5d-7df97750291f admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.880371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e7791-2099-4bb7-ac5d-7df97750291f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.880650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e7791-2099-4bb7-ac5d-7df97750291f admin admin] PATH_INFO: `/v3/projects/1cb7102375cf4f3683986dcaaa412a84/users/f5ce7e1612f3416cab11f5bda840571e/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.901255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e7791-2099-4bb7-ac5d-7df97750291f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb7102375cf4f3683986dcaaa412a84, user_id=f5ce7e1612f3416cab11f5bda840571e, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.904687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e7791-2099-4bb7-ac5d-7df97750291f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:00.932770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1129/2273] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:00 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:00.937776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59ee94-c6e1-4fdd-b5a1-4c3b94862147 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.027168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1145/2274] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:00 2022] PUT /identity/v3/projects/1cb7102375cf4f3683986dcaaa412a84/users/f5ce7e1612f3416cab11f5bda840571e/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:01.041738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2214e9-a6f6-4fa5-a1ce-18ec5b652be8 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.042200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2214e9-a6f6-4fa5-a1ce-18ec5b652be8 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.042519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2214e9-a6f6-4fa5-a1ce-18ec5b652be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.054209 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59ee94-c6e1-4fdd-b5a1-4c3b94862147 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.090470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-aa2214e9-a6f6-4fa5-a1ce-18ec5b652be8 None None] MFA Rules not processed for user `f5ce7e1612f3416cab11f5bda840571e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:01.091688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-aa2214e9-a6f6-4fa5-a1ce-18ec5b652be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:01.133996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:01.150534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794fc01-824a-471e-9a43-ad7784eec1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.151363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794fc01-824a-471e-9a43-ad7784eec1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.151639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794fc01-824a-471e-9a43-ad7784eec1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.151961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794fc01-824a-471e-9a43-ad7784eec1cd admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.153173 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794fc01-824a-471e-9a43-ad7784eec1cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.155252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794fc01-824a-471e-9a43-ad7784eec1cd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.158730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:01.178303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1146/2275] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:52:01.190022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2214e9-a6f6-4fa5-a1ce-18ec5b652be8 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.191489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-aa2214e9-a6f6-4fa5-a1ce-18ec5b652be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:01.260743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1130/2276] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:00 2022] POST /identity/v3/users => generated 295 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:52:01.264806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2214e9-a6f6-4fa5-a1ce-18ec5b652be8 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.266091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794fc01-824a-471e-9a43-ad7784eec1cd admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.269137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b2573-f71b-4335-a11a-6cd70d90dfe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.269757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b2573-f71b-4335-a11a-6cd70d90dfe5 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.270070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b2573-f71b-4335-a11a-6cd70d90dfe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.270370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b2573-f71b-4335-a11a-6cd70d90dfe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.281134 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794fc01-824a-471e-9a43-ad7784eec1cd admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.284731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b2573-f71b-4335-a11a-6cd70d90dfe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.286671 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776d21-db8f-41fc-b595-e1935ce76264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.287386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776d21-db8f-41fc-b595-e1935ce76264 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.287592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776d21-db8f-41fc-b595-e1935ce76264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.287930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776d21-db8f-41fc-b595-e1935ce76264 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.288281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b2573-f71b-4335-a11a-6cd70d90dfe5 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.288540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30776d21-db8f-41fc-b595-e1935ce76264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.290295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30776d21-db8f-41fc-b595-e1935ce76264 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.302351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1147/2277] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:01.306345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1131/2278] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:52:01.309612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b2573-f71b-4335-a11a-6cd70d90dfe5 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.321741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b2573-f71b-4335-a11a-6cd70d90dfe5 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.327231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e9cb5-beff-47f8-b5c8-3b49ab84b612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.329319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e9cb5-beff-47f8-b5c8-3b49ab84b612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.329319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e9cb5-beff-47f8-b5c8-3b49ab84b612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.329319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e9cb5-beff-47f8-b5c8-3b49ab84b612 admin admin] PATH_INFO: `/v3/projects/b93845e04a89497cb5f963884d95c2ba/users/b41cbf038c334a2fb1f7a73aebcf19c4/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.345265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e9cb5-beff-47f8-b5c8-3b49ab84b612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93845e04a89497cb5f963884d95c2ba, user_id=b41cbf038c334a2fb1f7a73aebcf19c4, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.348174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e9cb5-beff-47f8-b5c8-3b49ab84b612 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.468881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1148/2279] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:01 2022] PUT /identity/v3/projects/b93845e04a89497cb5f963884d95c2ba/users/b41cbf038c334a2fb1f7a73aebcf19c4/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:01.473191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776d21-db8f-41fc-b595-e1935ce76264 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.486265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776d21-db8f-41fc-b595-e1935ce76264 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.571538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86ed8f-0203-479e-a42a-7cc69d1f9702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.572240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86ed8f-0203-479e-a42a-7cc69d1f9702 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.572626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86ed8f-0203-479e-a42a-7cc69d1f9702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.573081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86ed8f-0203-479e-a42a-7cc69d1f9702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.573764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86ed8f-0203-479e-a42a-7cc69d1f9702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.575602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86ed8f-0203-479e-a42a-7cc69d1f9702 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.588004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1132/2280] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:01.592961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e9cb5-beff-47f8-b5c8-3b49ab84b612 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.604621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e9cb5-beff-47f8-b5c8-3b49ab84b612 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.616206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.617008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.617426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.618193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] PATH_INFO: `/v3/projects/b93845e04a89497cb5f963884d95c2ba/users/b41cbf038c334a2fb1f7a73aebcf19c4/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.621005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93845e04a89497cb5f963884d95c2ba, user_id=b41cbf038c334a2fb1f7a73aebcf19c4, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.624027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.636143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1149/2281] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:01 2022] PUT /identity/v3/projects/b93845e04a89497cb5f963884d95c2ba/users/b41cbf038c334a2fb1f7a73aebcf19c4/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:08.569532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a91196-8eb8-4346-a6d7-38eadc783181 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:08.570467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a91196-8eb8-4346-a6d7-38eadc783181 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:08.570467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a91196-8eb8-4346-a6d7-38eadc783181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:08.596808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-92a91196-8eb8-4346-a6d7-38eadc783181 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:08.598547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-92a91196-8eb8-4346-a6d7-38eadc783181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:08.707536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1133/2282] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:52:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:08.726519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:08.728516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:08.903689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b484f8-13f8-4db3-89da-404c280f7d1e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:08.919395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:08.920390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:08.921160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:08.921160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:08.933275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:08.935705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:08.948079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1150/2283] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:52:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:52:09.236509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d30d5-940a-4897-8284-3121e594bcb1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:09.237045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d30d5-940a-4897-8284-3121e594bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:09.237330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d30d5-940a-4897-8284-3121e594bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:09.282838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7f4d30d5-940a-4897-8284-3121e594bcb1 None None] MFA Rules not processed for user `b41cbf038c334a2fb1f7a73aebcf19c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:09.285421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7f4d30d5-940a-4897-8284-3121e594bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:09.418970 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:09.443685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1134/2284] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:09.456452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:09.458851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:09.560453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:09.617692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabeb6f-4a9a-4702-affc-bb013ec3fd24 service swift] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:09.633345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1d96a-7b49-4646-809d-6137d5f3de5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:09.633644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d96a-7b49-4646-809d-6137d5f3de5c service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:09.634217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d96a-7b49-4646-809d-6137d5f3de5c service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:09.636191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d96a-7b49-4646-809d-6137d5f3de5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:09.844364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1d96a-7b49-4646-809d-6137d5f3de5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:09.846052 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1d96a-7b49-4646-809d-6137d5f3de5c service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:09.873399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1151/2285] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:13.220152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d30d5-940a-4897-8284-3121e594bcb1 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:13.350773 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d30d5-940a-4897-8284-3121e594bcb1 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:13.355488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:13.356237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:13.356572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:13.356844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] PATH_INFO: `/v3/users/f5ce7e1612f3416cab11f5bda840571e` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:13.358109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ce7e1612f3416cab11f5bda840571e)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:13.360047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:13.742694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ce7e1612f3416cab11f5bda840571e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:13.863177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ce7e1612f3416cab11f5bda840571e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:13.865389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ce7e1612f3416cab11f5bda840571e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:13.893388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ce7e1612f3416cab11f5bda840571e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:13.895597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ce7e1612f3416cab11f5bda840571e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:13.959534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1135/2286] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:52:13 2022] DELETE /identity/v3/users/f5ce7e1612f3416cab11f5bda840571e => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:13.965169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b44c3-b2cf-4167-b1df-7951cea1ff83 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:13.966288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b44c3-b2cf-4167-b1df-7951cea1ff83 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:13.967053 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b44c3-b2cf-4167-b1df-7951cea1ff83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:14.008373 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-645b44c3-b2cf-4167-b1df-7951cea1ff83 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:14.013011 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-645b44c3-b2cf-4167-b1df-7951cea1ff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:14.176568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1152/2287] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:14.188259 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:14.193013 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:14.274638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e5664-f2a6-4e2b-a412-5e62d5e7e469 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:14.280772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9860cb-3c7d-458d-b739-1a292e599fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:14.281495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9860cb-3c7d-458d-b739-1a292e599fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:14.281979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9860cb-3c7d-458d-b739-1a292e599fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:14.282331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9860cb-3c7d-458d-b739-1a292e599fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:14.295420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9860cb-3c7d-458d-b739-1a292e599fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:14.296882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9860cb-3c7d-458d-b739-1a292e599fd6 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:14.310284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1136/2288] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:14.901179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b44c3-b2cf-4167-b1df-7951cea1ff83 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:14.905250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-645b44c3-b2cf-4167-b1df-7951cea1ff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:14.934638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b44c3-b2cf-4167-b1df-7951cea1ff83 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:14.948147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:14.951137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:14.951559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:14.952943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] PATH_INFO: `/v3/projects/1cb7102375cf4f3683986dcaaa412a84` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:14.959594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cb7102375cf4f3683986dcaaa412a84)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:14.963689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:15.143594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cb7102375cf4f3683986dcaaa412a84', 'request_id': 'req-227524fe-c296-4221-ab15-b1b212ad4433'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:15.250910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cb7102375cf4f3683986dcaaa412a84', 'request_id': 'req-227524fe-c296-4221-ab15-b1b212ad4433'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:15.266736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cb7102375cf4f3683986dcaaa412a84', 'request_id': 'req-227524fe-c296-4221-ab15-b1b212ad4433'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:15.289038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] The token cache is being invalidate because project 1cb7102375cf4f3683986dcaaa412a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:52:15.289487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:15.290722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-227524fe-c296-4221-ab15-b1b212ad4433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:15.293678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1153/2289] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:52:14 2022] DELETE /identity/v3/projects/1cb7102375cf4f3683986dcaaa412a84 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:15.298598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995ddde-2f59-44f0-88f9-65675540e18d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:15.299150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995ddde-2f59-44f0-88f9-65675540e18d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:15.299285 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995ddde-2f59-44f0-88f9-65675540e18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:15.340766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-4995ddde-2f59-44f0-88f9-65675540e18d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:15.342533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4995ddde-2f59-44f0-88f9-65675540e18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:15.366716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:15.473905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1137/2290] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:15.482552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539f1a-30e2-4b3b-8ed5-5407664c6c3a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:15.483105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539f1a-30e2-4b3b-8ed5-5407664c6c3a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:15.483637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539f1a-30e2-4b3b-8ed5-5407664c6c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:15.504349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:15.516093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d6539f1a-30e2-4b3b-8ed5-5407664c6c3a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:15.518133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d6539f1a-30e2-4b3b-8ed5-5407664c6c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:15.528372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1154/2291] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:15.532960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995ddde-2f59-44f0-88f9-65675540e18d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:15.546859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995ddde-2f59-44f0-88f9-65675540e18d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:15.553583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:15.554584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:15.561213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:15.561213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:15.561213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:52:15.561213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:15.561213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:15.575293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1138/2292] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:52:15.580983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6539f1a-30e2-4b3b-8ed5-5407664c6c3a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:15.598222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6539f1a-30e2-4b3b-8ed5-5407664c6c3a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:15.604058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bbcad-91bb-4d02-9ac4-10c5eb752cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:15.606607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bbcad-91bb-4d02-9ac4-10c5eb752cac admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:15.606914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bbcad-91bb-4d02-9ac4-10c5eb752cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:15.610461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bbcad-91bb-4d02-9ac4-10c5eb752cac admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:15.611546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bbcad-91bb-4d02-9ac4-10c5eb752cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:15.613949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bbcad-91bb-4d02-9ac4-10c5eb752cac admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:15.722159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1155/2293] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:15 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:15.726520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:15.740750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cdf85-7815-45c7-9801-741de9ebab76 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:15.857355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa7e93-27cf-4c55-9463-70478d212051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:15.858069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa7e93-27cf-4c55-9463-70478d212051 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:15.858357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa7e93-27cf-4c55-9463-70478d212051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:15.858621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa7e93-27cf-4c55-9463-70478d212051 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:15.859307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa7e93-27cf-4c55-9463-70478d212051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:15.861843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa7e93-27cf-4c55-9463-70478d212051 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:16.045620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1139/2294] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:15 2022] POST /identity/v3/users => generated 286 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:52:16.049843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bbcad-91bb-4d02-9ac4-10c5eb752cac admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:16.063969 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bbcad-91bb-4d02-9ac4-10c5eb752cac admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:16.069954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e8ea7-1010-4022-9af3-2baf685f538a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:16.070892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8ea7-1010-4022-9af3-2baf685f538a admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:16.071240 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8ea7-1010-4022-9af3-2baf685f538a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:16.071675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8ea7-1010-4022-9af3-2baf685f538a admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:16.080379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e8ea7-1010-4022-9af3-2baf685f538a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:16.082311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e8ea7-1010-4022-9af3-2baf685f538a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:16.109720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1156/2295] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:52:16.116061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa7e93-27cf-4c55-9463-70478d212051 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:16.130014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa7e93-27cf-4c55-9463-70478d212051 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:16.143050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fee08-14bc-41f4-b680-9bfd2c8c8754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:16.143050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fee08-14bc-41f4-b680-9bfd2c8c8754 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:16.143050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fee08-14bc-41f4-b680-9bfd2c8c8754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:16.143050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fee08-14bc-41f4-b680-9bfd2c8c8754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:16.143561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fee08-14bc-41f4-b680-9bfd2c8c8754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:16.143561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fee08-14bc-41f4-b680-9bfd2c8c8754 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:16.165065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1140/2296] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:16.168624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e8ea7-1010-4022-9af3-2baf685f538a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:16.181607 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e8ea7-1010-4022-9af3-2baf685f538a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:16.187695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:16.188465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:16.188978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:16.189453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] PATH_INFO: `/v3/projects/6e895dd83dbf4458b7d6efaace0d209a/users/f08b2eba50904dda89fb743c9cf28958/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:16.214162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e895dd83dbf4458b7d6efaace0d209a, user_id=f08b2eba50904dda89fb743c9cf28958, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:16.218085 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:16.318295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1157/2297] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:16 2022] PUT /identity/v3/projects/6e895dd83dbf4458b7d6efaace0d209a/users/f08b2eba50904dda89fb743c9cf28958/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:16.320354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41727f5-8daa-4f69-94bd-1e7d8b18cf17 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:16.321172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41727f5-8daa-4f69-94bd-1e7d8b18cf17 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:16.321453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41727f5-8daa-4f69-94bd-1e7d8b18cf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:16.351149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d41727f5-8daa-4f69-94bd-1e7d8b18cf17 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:16.351149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d41727f5-8daa-4f69-94bd-1e7d8b18cf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:16.444823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1141/2298] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:52:16.455598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:16.457911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:16.552252 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815194b-5b81-4569-bdfa-0f982098a771 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:16.556294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:16.556921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:16.557207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:16.557499 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:16.571626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:16.573782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:16.585246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1158/2299] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:18.457732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b01a-28a5-44dd-803e-396314ee8804 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:18.459176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b01a-28a5-44dd-803e-396314ee8804 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:18.459176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b01a-28a5-44dd-803e-396314ee8804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:18.499742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-23b9b01a-28a5-44dd-803e-396314ee8804 None None] MFA Rules not processed for user `f08b2eba50904dda89fb743c9cf28958`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:18.501040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-23b9b01a-28a5-44dd-803e-396314ee8804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:18.527697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:18.622383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1142/2300] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:52:18.635356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:18.637781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:18.725200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e4142-3ab9-4dce-9d06-a83fdbf59974 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:18.728946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:18.730450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:18.730742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:18.731016 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:18.754684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:18.756858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:18.769372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1159/2301] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:20.847655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427570-4657-49d0-b347-26e6a37c9a9e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:20.850198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427570-4657-49d0-b347-26e6a37c9a9e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:20.850198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427570-4657-49d0-b347-26e6a37c9a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:20.868362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2b427570-4657-49d0-b347-26e6a37c9a9e None None] MFA Rules not processed for user `f08b2eba50904dda89fb743c9cf28958`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:20.869596 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2b427570-4657-49d0-b347-26e6a37c9a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:20.877551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1143/2302] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:24.886612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:24.888181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:24.943322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:24.964464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09ff0c-5ee5-4993-8afb-29e4ef7abc4f service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:24.969881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab14f8-79b9-4c98-85f7-5197bbda3c88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:24.970526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab14f8-79b9-4c98-85f7-5197bbda3c88 service placement] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:24.970809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab14f8-79b9-4c98-85f7-5197bbda3c88 service placement] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:24.971087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab14f8-79b9-4c98-85f7-5197bbda3c88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:24.972926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-75ab14f8-79b9-4c98-85f7-5197bbda3c88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:24.985561 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab14f8-79b9-4c98-85f7-5197bbda3c88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:24.987189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab14f8-79b9-4c98-85f7-5197bbda3c88 service placement] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:24.997984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1160/2303] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:30.862904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=25 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:34.586630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db58e1-ced8-442a-ae51-ef5a4db01a53 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:34.587371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db58e1-ced8-442a-ae51-ef5a4db01a53 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:34.588047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db58e1-ced8-442a-ae51-ef5a4db01a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:34.663843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b5db58e1-ced8-442a-ae51-ef5a4db01a53 None None] MFA Rules not processed for user `f19bf0e6a8ad425cbbf804d6d7be5a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:34.665381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b5db58e1-ced8-442a-ae51-ef5a4db01a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:34.741448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1144/2304] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:35.443604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=10 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:38.346221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2bdb-064b-4a41-adff-46a02cc4440a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:38.346849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2bdb-064b-4a41-adff-46a02cc4440a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:38.347234 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2bdb-064b-4a41-adff-46a02cc4440a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:38.380153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-827c2bdb-064b-4a41-adff-46a02cc4440a None None] MFA Rules not processed for user `63ff0ea67ee64e14aafc5c51cad9684f`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:38.381464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-827c2bdb-064b-4a41-adff-46a02cc4440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:38.458865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1161/2305] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:52:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:39.304656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0d553-ce0c-41ed-82e3-0b428a0edf25 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:39.305114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0d553-ce0c-41ed-82e3-0b428a0edf25 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:39.305358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0d553-ce0c-41ed-82e3-0b428a0edf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:39.331053 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-47d0d553-ce0c-41ed-82e3-0b428a0edf25 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:39.332627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-47d0d553-ce0c-41ed-82e3-0b428a0edf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:39.428889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1145/2306] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 17:52:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:39.464696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c2bdb-064b-4a41-adff-46a02cc4440a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:39.467631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-827c2bdb-064b-4a41-adff-46a02cc4440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:39.592450 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c2bdb-064b-4a41-adff-46a02cc4440a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:39.598484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f450d2-cb1f-442f-b573-2f28b38d13fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:39.599174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f450d2-cb1f-442f-b573-2f28b38d13fa service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:39.599739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f450d2-cb1f-442f-b573-2f28b38d13fa service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:39.600226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f450d2-cb1f-442f-b573-2f28b38d13fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:39.612448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f450d2-cb1f-442f-b573-2f28b38d13fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:39.614130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f450d2-cb1f-442f-b573-2f28b38d13fa service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:39.629351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1162/2307] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:52:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:52:49.419837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=50 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:49.621799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=57 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:06.694992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e307b8-c0bc-4cc5-9134-16a7e0cb4af7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:06.702667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e307b8-c0bc-4cc5-9134-16a7e0cb4af7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:06.702667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e307b8-c0bc-4cc5-9134-16a7e0cb4af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:06.723661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-03e307b8-c0bc-4cc5-9134-16a7e0cb4af7 None None] MFA Rules not processed for user `f08b2eba50904dda89fb743c9cf28958`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:06.725316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-03e307b8-c0bc-4cc5-9134-16a7e0cb4af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:06.733193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1146/2308] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:53:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:13.611235 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f450d2-cb1f-442f-b573-2f28b38d13fa service swift] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:13.625775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f450d2-cb1f-442f-b573-2f28b38d13fa service swift] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:13.659548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1bf09-0441-4b25-82e0-752728d81092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:13.661466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1bf09-0441-4b25-82e0-752728d81092 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:13.661605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1bf09-0441-4b25-82e0-752728d81092 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:13.668807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1bf09-0441-4b25-82e0-752728d81092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:13.678486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-81b1bf09-0441-4b25-82e0-752728d81092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:13.774835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1bf09-0441-4b25-82e0-752728d81092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:13.779055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1bf09-0441-4b25-82e0-752728d81092 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:13.800340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1163/2309] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:16.716865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:17.984698 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a4698-5e05-4652-84f1-4d33379aee0b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:17.985347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a4698-5e05-4652-84f1-4d33379aee0b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:17.985347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a4698-5e05-4652-84f1-4d33379aee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:18.017922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-341a4698-5e05-4652-84f1-4d33379aee0b None None] MFA Rules not processed for user `251bc85f89774142a97acd77895b14e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:18.019630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-341a4698-5e05-4652-84f1-4d33379aee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:18.113986 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1147/2310] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:53:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:18.398637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8d5bc-c611-4736-b3c3-700ea5602e4a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:18.399813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8d5bc-c611-4736-b3c3-700ea5602e4a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:18.400341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8d5bc-c611-4736-b3c3-700ea5602e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:18.423624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-17c8d5bc-c611-4736-b3c3-700ea5602e4a None None] MFA Rules not processed for user `7ca82f3632bc404691835a2db6167939`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:18.425254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-17c8d5bc-c611-4736-b3c3-700ea5602e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:18.500980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1164/2311] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:53:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:28.097041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=25 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:28.490404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=55 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:44.318551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d0e22-d381-4344-90b3-14f2c8578d5e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:44.319306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d0e22-d381-4344-90b3-14f2c8578d5e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:44.319306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d0e22-d381-4344-90b3-14f2c8578d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:44.340356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d34d0e22-d381-4344-90b3-14f2c8578d5e None None] MFA Rules not processed for user `612dfebb8ebc401bb81d3d3333577566`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:44.341778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d34d0e22-d381-4344-90b3-14f2c8578d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:44.409613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1148/2312] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:47.069794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bfea-5306-4f3a-9c5a-ed76222f636e None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:47.070411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bfea-5306-4f3a-9c5a-ed76222f636e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:47.070411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bfea-5306-4f3a-9c5a-ed76222f636e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:47.092346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c0d4bfea-5306-4f3a-9c5a-ed76222f636e None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:47.093555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c0d4bfea-5306-4f3a-9c5a-ed76222f636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:47.101135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1165/2313] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:53:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:47.243262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d0e22-d381-4344-90b3-14f2c8578d5e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:47.254443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d0e22-d381-4344-90b3-14f2c8578d5e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:47.259440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9088e05-9613-4f23-9811-e2b8387ee588 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:47.260045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9088e05-9613-4f23-9811-e2b8387ee588 service swift] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:47.260324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9088e05-9613-4f23-9811-e2b8387ee588 service swift] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:47.260740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9088e05-9613-4f23-9811-e2b8387ee588 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:47.270117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9088e05-9613-4f23-9811-e2b8387ee588 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:47.271762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9088e05-9613-4f23-9811-e2b8387ee588 service swift] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:47.281959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1149/2314] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:53:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:53:57.086068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:57.278075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=31 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:05.638618 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4bfea-5306-4f3a-9c5a-ed76222f636e None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:05.640201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c0d4bfea-5306-4f3a-9c5a-ed76222f636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:05.652785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4bfea-5306-4f3a-9c5a-ed76222f636e None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:05.656516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:05.657127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:05.657407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:05.657681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:05.659497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:05.745387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:05.747274 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:05.759690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1166/2315] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:06.342130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e4a1-d3fe-4a68-9e38-4f73b1a6510c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:06.342754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e4a1-d3fe-4a68-9e38-4f73b1a6510c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:06.342903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e4a1-d3fe-4a68-9e38-4f73b1a6510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:06.365614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-d145e4a1-d3fe-4a68-9e38-4f73b1a6510c None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:06.367044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d145e4a1-d3fe-4a68-9e38-4f73b1a6510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:06.374800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1150/2316] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 17:54:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:06.388468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:06.402702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2943d37-153c-4b84-afa3-3dbb28ba0442 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:06.407804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbaa71-e39a-4d6b-b955-c118a553a8be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:06.408407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbaa71-e39a-4d6b-b955-c118a553a8be service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:06.408687 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbaa71-e39a-4d6b-b955-c118a553a8be service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:06.408958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbaa71-e39a-4d6b-b955-c118a553a8be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:06.417677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbaa71-e39a-4d6b-b955-c118a553a8be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:06.419324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbaa71-e39a-4d6b-b955-c118a553a8be service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:06.428074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1167/2317] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:54:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:54:06.959404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e4a1-d3fe-4a68-9e38-4f73b1a6510c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:06.972583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e4a1-d3fe-4a68-9e38-4f73b1a6510c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:06.976280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:06.976889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:06.977219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:06.977526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:06.980005 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:06.994506 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:06.996245 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:07.008086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1151/2318] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:09.749612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbaa71-e39a-4d6b-b955-c118a553a8be service swift] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:09.761542 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbaa71-e39a-4d6b-b955-c118a553a8be service swift] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:09.766263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:09.766901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:09.767181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:09.767462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] PATH_INFO: `/v3/users/f08b2eba50904dda89fb743c9cf28958` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:09.768627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08b2eba50904dda89fb743c9cf28958)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:09.770589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:10.047127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08b2eba50904dda89fb743c9cf28958'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.061621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08b2eba50904dda89fb743c9cf28958'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.148528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08b2eba50904dda89fb743c9cf28958'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.149171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08b2eba50904dda89fb743c9cf28958'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.149813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08b2eba50904dda89fb743c9cf28958'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.163985 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1168/2319] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:54:09 2022] DELETE /identity/v3/users/f08b2eba50904dda89fb743c9cf28958 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:54:10.433268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:10.444675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c13a1-e923-4eb4-898e-5487711135d3 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:10.533231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:10.533931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:10.534216 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:10.534489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] PATH_INFO: `/v3/projects/6e895dd83dbf4458b7d6efaace0d209a` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:10.535644 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e895dd83dbf4458b7d6efaace0d209a)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:10.537603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:10.679334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e895dd83dbf4458b7d6efaace0d209a', 'request_id': 'req-c971d8a6-b262-4445-bd8d-d027f40e4086'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.762766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e895dd83dbf4458b7d6efaace0d209a', 'request_id': 'req-c971d8a6-b262-4445-bd8d-d027f40e4086'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.770846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e895dd83dbf4458b7d6efaace0d209a', 'request_id': 'req-c971d8a6-b262-4445-bd8d-d027f40e4086'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.794219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] The token cache is being invalidate because project 6e895dd83dbf4458b7d6efaace0d209a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:54:10.794655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.795351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c971d8a6-b262-4445-bd8d-d027f40e4086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:10.797169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1152/2320] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:54:10 2022] DELETE /identity/v3/projects/6e895dd83dbf4458b7d6efaace0d209a => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:54:20.159822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:20.781090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:23.220708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:23.222461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:23.302315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeff253-00f4-451c-9eac-21b9da39fd19 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:23.374496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:23.375117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:23.375395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:23.375670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:23.377546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:23.444297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:23.446126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:23.456931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1169/2321] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:30.444761 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ecb37-7dc0-4a38-af29-09e828b62c59 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:30.445420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ecb37-7dc0-4a38-af29-09e828b62c59 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:30.445420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ecb37-7dc0-4a38-af29-09e828b62c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:30.467778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6a3ecb37-7dc0-4a38-af29-09e828b62c59 None None] MFA Rules not processed for user `2b6b0d0cbdb54a46a7e9d6936c47cc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:30.472377 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6a3ecb37-7dc0-4a38-af29-09e828b62c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:30.481170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1153/2322] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:54:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:54:30.824345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:30.826026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:30.868421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:30.885440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffe33e-ab67-41a4-9490-7ac11a813368 service nova] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:30.889415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:30.895356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:30.895356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:30.895356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:30.895356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:30.902503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:30.903928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:30.914543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1170/2323] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:31.468415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ecb37-7dc0-4a38-af29-09e828b62c59 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:31.472206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6a3ecb37-7dc0-4a38-af29-09e828b62c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:31.544772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ecb37-7dc0-4a38-af29-09e828b62c59 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:31.550151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce598ce-d896-4d3c-befa-a5c63e3a3190 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:31.550772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce598ce-d896-4d3c-befa-a5c63e3a3190 service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:31.551141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce598ce-d896-4d3c-befa-a5c63e3a3190 service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:31.551423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce598ce-d896-4d3c-befa-a5c63e3a3190 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:31.553268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8ce598ce-d896-4d3c-befa-a5c63e3a3190 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:31.566192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce598ce-d896-4d3c-befa-a5c63e3a3190 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:31.567846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce598ce-d896-4d3c-befa-a5c63e3a3190 service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:31.579810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1154/2324] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:32.958979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:32.960461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:32.972606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d8eb2-b160-4747-9311-3b65343be927 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:32.977204 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:32.977823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:32.978144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:32.978421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:32.980268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:32.991839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:32.993303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:33.004650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1171/2325] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:34.802030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742077ab-377e-47a3-9fbf-c3e1258cedf1 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:34.802789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742077ab-377e-47a3-9fbf-c3e1258cedf1 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:34.802789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742077ab-377e-47a3-9fbf-c3e1258cedf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:34.854898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-742077ab-377e-47a3-9fbf-c3e1258cedf1 None None] MFA Rules not processed for user `63ff0ea67ee64e14aafc5c51cad9684f`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:34.856045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-742077ab-377e-47a3-9fbf-c3e1258cedf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:34.923905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1155/2326] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:54:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:36.394886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:36.396715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:36.413554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0636-9a84-4286-aaee-06b286c563ff service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:36.419265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:36.420141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:36.420585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:36.421729 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:36.422745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:36.494070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:36.495889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:36.507026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1172/2327] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:36.806996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742077ab-377e-47a3-9fbf-c3e1258cedf1 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:36.820613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742077ab-377e-47a3-9fbf-c3e1258cedf1 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:36.830588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:36.830588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:36.830588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:36.830588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:36.832081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:36.845272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:36.847307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:36.859081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1156/2328] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:37.957435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:37.969597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be8ca7-30bd-4961-990a-3adf3b4c282c service nova] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:37.975689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:37.976311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:37.976597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:37.976874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:37.978734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:37.994342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:37.994342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:38.006102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1173/2329] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:38.105838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:38.107697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:38.175305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0661ed-683f-46a0-9b92-a8a6413759ec service nova] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:38.180231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:38.180909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:38.181193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:38.181470 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] PATH_INFO: `/v3/users/7ca82f3632bc404691835a2db6167939` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:38.182633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca82f3632bc404691835a2db6167939)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:38.184696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:38.202489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:38.383085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca82f3632bc404691835a2db6167939'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:38.470549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca82f3632bc404691835a2db6167939'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:38.471890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca82f3632bc404691835a2db6167939'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:38.481102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca82f3632bc404691835a2db6167939'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:38.483447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca82f3632bc404691835a2db6167939'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:38.519988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1157/2330] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:54:38 2022] DELETE /identity/v3/users/7ca82f3632bc404691835a2db6167939 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:54:38.871376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:38.884256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196066ff-8aa7-4711-9546-3427f1998723 service placement] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:38.954221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:38.954858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:38.955144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:38.955417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] PATH_INFO: `/v3/projects/4fa9fd072c704af2950a9bb4339d374d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:38.956567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa9fd072c704af2950a9bb4339d374d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:38.958537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:39.106646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa9fd072c704af2950a9bb4339d374d', 'request_id': 'req-00a7268c-82b5-4aff-b74d-bf8c08538511'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:39.109786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:39.207318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa9fd072c704af2950a9bb4339d374d', 'request_id': 'req-00a7268c-82b5-4aff-b74d-bf8c08538511'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:39.220631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa9fd072c704af2950a9bb4339d374d', 'request_id': 'req-00a7268c-82b5-4aff-b74d-bf8c08538511'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:39.243594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] The token cache is being invalidate because project 4fa9fd072c704af2950a9bb4339d374d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:54:39.243896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:39.244583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-00a7268c-82b5-4aff-b74d-bf8c08538511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:39.246386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1174/2331] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:54:38 2022] DELETE /identity/v3/projects/4fa9fd072c704af2950a9bb4339d374d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:54:48.187555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:48.189670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:48.276067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b616e-af7b-4ef1-a895-357ff8976eca admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:48.359827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:48.360372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:48.360658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:48.360936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:48.362893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:48.400577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:48.434036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:48.435829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:48.446218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1158/2332] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:49.366001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=3,keystone:SELECT=6 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:59.241919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=14 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:59.791483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41aae-a6e1-4201-ad4c-2f0d95758b4c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:59.792113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41aae-a6e1-4201-ad4c-2f0d95758b4c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:59.792113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41aae-a6e1-4201-ad4c-2f0d95758b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:59.813704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1cb41aae-a6e1-4201-ad4c-2f0d95758b4c None None] MFA Rules not processed for user `251bc85f89774142a97acd77895b14e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:59.815009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1cb41aae-a6e1-4201-ad4c-2f0d95758b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:59.887223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1175/2333] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:54:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:00.314791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:00.316318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:00.398885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430edde-820e-4a1f-a31f-bf20a5a302a7 service nova] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:00.403408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:00.404599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:00.404939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:00.405162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:00.407166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:00.417760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:00.419825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:00.430376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1159/2334] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:06.370058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f415c-3886-4502-a1ca-63e47caba878 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:06.370716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f415c-3886-4502-a1ca-63e47caba878 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:06.370716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f415c-3886-4502-a1ca-63e47caba878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:06.391879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-456f415c-3886-4502-a1ca-63e47caba878 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:06.393103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-456f415c-3886-4502-a1ca-63e47caba878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:06.464606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1176/2335] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:55:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:06.651336 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:06.653027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:06.741964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be01e0-ed2f-4e20-a3ef-8d8fe507fc39 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:06.752457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:06.753399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:06.753727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:06.754074 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:06.764207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:06.765884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:06.776403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1160/2336] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:55:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:55:07.452458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f415c-3886-4502-a1ca-63e47caba878 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:07.453976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-456f415c-3886-4502-a1ca-63e47caba878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:07.472125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f415c-3886-4502-a1ca-63e47caba878 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:07.478357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cd6bc-a8d1-42af-9a5b-6d593213206a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:07.479293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cd6bc-a8d1-42af-9a5b-6d593213206a service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:07.479647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cd6bc-a8d1-42af-9a5b-6d593213206a service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:07.479920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cd6bc-a8d1-42af-9a5b-6d593213206a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:07.482129 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-085cd6bc-a8d1-42af-9a5b-6d593213206a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:07.496508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cd6bc-a8d1-42af-9a5b-6d593213206a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:07.498899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cd6bc-a8d1-42af-9a5b-6d593213206a service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:07.511871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1177/2337] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:16.771924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=58 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:17.506226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=56 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:21.359937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:21.363319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:21.380153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414aa86b-544b-4bb1-8c01-37a1d38feae5 service swift] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:21.387624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae9782-0102-45ac-835b-57d1645ee869 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:21.398915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae9782-0102-45ac-835b-57d1645ee869 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:21.398915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae9782-0102-45ac-835b-57d1645ee869 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:21.398915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae9782-0102-45ac-835b-57d1645ee869 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:21.398915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a6ae9782-0102-45ac-835b-57d1645ee869 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:21.407061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae9782-0102-45ac-835b-57d1645ee869 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:21.408791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae9782-0102-45ac-835b-57d1645ee869 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:21.425782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1161/2338] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:31.416832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:31.707174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4486a9-c7bb-462e-83ba-9d452d0720d3 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:31.708786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4486a9-c7bb-462e-83ba-9d452d0720d3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:31.708786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4486a9-c7bb-462e-83ba-9d452d0720d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:31.731048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3b4486a9-c7bb-462e-83ba-9d452d0720d3 None None] MFA Rules not processed for user `f19bf0e6a8ad425cbbf804d6d7be5a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:31.732421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3b4486a9-c7bb-462e-83ba-9d452d0720d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:31.839289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1178/2339] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:32.668777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373e20-ab9d-4920-a324-5b080929df19 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:32.669633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373e20-ab9d-4920-a324-5b080929df19 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:32.669809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373e20-ab9d-4920-a324-5b080929df19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:32.698728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-32373e20-ab9d-4920-a324-5b080929df19 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:32.701993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-32373e20-ab9d-4920-a324-5b080929df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:32.730900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1162/2340] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:55:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:32.732767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4486a9-c7bb-462e-83ba-9d452d0720d3 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:32.746236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4486a9-c7bb-462e-83ba-9d452d0720d3 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:32.754894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e5971-831a-4e20-a235-119f00eefcef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:32.754894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e5971-831a-4e20-a235-119f00eefcef service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:32.754894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e5971-831a-4e20-a235-119f00eefcef service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:32.754894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e5971-831a-4e20-a235-119f00eefcef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:32.764246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e5971-831a-4e20-a235-119f00eefcef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:32.766075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e5971-831a-4e20-a235-119f00eefcef service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:32.775308 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1179/2341] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:55:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:55:42.689082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:42.771689 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=31 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:48.736234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373e20-ab9d-4920-a324-5b080929df19 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.748268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373e20-ab9d-4920-a324-5b080929df19 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.753205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.759138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.759138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.759138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.759138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:48.840821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.844460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.853529 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1163/2342] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:49.550454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f07bf-3369-4028-8a78-91b60a85b940 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:49.554459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f07bf-3369-4028-8a78-91b60a85b940 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:49.554459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f07bf-3369-4028-8a78-91b60a85b940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:49.572762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-864f07bf-3369-4028-8a78-91b60a85b940 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:49.574267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-864f07bf-3369-4028-8a78-91b60a85b940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:49.588147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1180/2343] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 17:55:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:49.603129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:49.615700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2da7b-e8b3-4a0f-8c95-4c2934f3dab0 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:49.622501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d12f26-2a4b-4113-aef1-32843be5e6e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:49.623183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d12f26-2a4b-4113-aef1-32843be5e6e8 service swift] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:49.623538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d12f26-2a4b-4113-aef1-32843be5e6e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:49.623883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d12f26-2a4b-4113-aef1-32843be5e6e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:49.635522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d12f26-2a4b-4113-aef1-32843be5e6e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:49.638441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d12f26-2a4b-4113-aef1-32843be5e6e8 service swift] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:49.648806 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1164/2344] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:55:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:55:53.313329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f07bf-3369-4028-8a78-91b60a85b940 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:53.313329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-864f07bf-3369-4028-8a78-91b60a85b940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:53.332191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f07bf-3369-4028-8a78-91b60a85b940 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:53.338393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:53.339090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:53.339376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:53.339805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] PATH_INFO: `/v3/users/63ff0ea67ee64e14aafc5c51cad9684f` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:53.340998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ff0ea67ee64e14aafc5c51cad9684f)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:53.343181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:53.544507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ff0ea67ee64e14aafc5c51cad9684f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.560889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ff0ea67ee64e14aafc5c51cad9684f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.658919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ff0ea67ee64e14aafc5c51cad9684f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.659727 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ff0ea67ee64e14aafc5c51cad9684f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.660441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0b0404e2-61e7-4956-a36e-92140f3c333f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ff0ea67ee64e14aafc5c51cad9684f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.675808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1181/2345] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:53 2022] DELETE /identity/v3/users/63ff0ea67ee64e14aafc5c51cad9684f => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:55:53.937411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d12f26-2a4b-4113-aef1-32843be5e6e8 service swift] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:53.949185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d12f26-2a4b-4113-aef1-32843be5e6e8 service swift] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.032546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.033208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.033490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.033767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] PATH_INFO: `/v3/projects/151e9d2b290047e8acd9468c721b9dd3` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.035035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=151e9d2b290047e8acd9468c721b9dd3)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.037084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.143585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151e9d2b290047e8acd9468c721b9dd3', 'request_id': 'req-a8513189-1a93-4607-8cb5-ce565e712b93'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.233085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151e9d2b290047e8acd9468c721b9dd3', 'request_id': 'req-a8513189-1a93-4607-8cb5-ce565e712b93'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.239523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151e9d2b290047e8acd9468c721b9dd3', 'request_id': 'req-a8513189-1a93-4607-8cb5-ce565e712b93'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.251402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] The token cache is being invalidate because project 151e9d2b290047e8acd9468c721b9dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:54.251690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.252330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a8513189-1a93-4607-8cb5-ce565e712b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.253861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1165/2346] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:53 2022] DELETE /identity/v3/projects/151e9d2b290047e8acd9468c721b9dd3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:55:54.258980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e340-cef3-4c7a-be2e-1694384f69d3 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.259269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e340-cef3-4c7a-be2e-1694384f69d3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.259552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e340-cef3-4c7a-be2e-1694384f69d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.282950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-06b8e340-cef3-4c7a-be2e-1694384f69d3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:54.284309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-06b8e340-cef3-4c7a-be2e-1694384f69d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:54.357814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1182/2347] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:54.359583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460521f-1fb6-4875-8dff-487f78ef485c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.359906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460521f-1fb6-4875-8dff-487f78ef485c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.360185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460521f-1fb6-4875-8dff-487f78ef485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.380853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-1460521f-1fb6-4875-8dff-487f78ef485c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:54.382158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1460521f-1fb6-4875-8dff-487f78ef485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:54.391436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1166/2348] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:54.397352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e340-cef3-4c7a-be2e-1694384f69d3 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.408465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e340-cef3-4c7a-be2e-1694384f69d3 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.413200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.413930 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.414214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.414487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.415141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:55:54.415444 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.417248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.429666 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1183/2349] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:55:54.434051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460521f-1fb6-4875-8dff-487f78ef485c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.440078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:54.444642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460521f-1fb6-4875-8dff-487f78ef485c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.450233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dfb60-36be-4461-9668-a8b11fa3252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.450818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dfb60-36be-4461-9668-a8b11fa3252f admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.451093 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dfb60-36be-4461-9668-a8b11fa3252f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.451488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dfb60-36be-4461-9668-a8b11fa3252f admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.452129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dfb60-36be-4461-9668-a8b11fa3252f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.454088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dfb60-36be-4461-9668-a8b11fa3252f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.579169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1167/2350] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:54.583542 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.594181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5a90-395f-494f-a1cb-f9aede04021c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.645161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:54.654141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101fee6-b257-4358-b6c5-7ae73bf09408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.655015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101fee6-b257-4358-b6c5-7ae73bf09408 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.655294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101fee6-b257-4358-b6c5-7ae73bf09408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.655574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101fee6-b257-4358-b6c5-7ae73bf09408 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.656220 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101fee6-b257-4358-b6c5-7ae73bf09408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.658208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101fee6-b257-4358-b6c5-7ae73bf09408 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.742665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1184/2351] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:54.747799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dfb60-36be-4461-9668-a8b11fa3252f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.768655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dfb60-36be-4461-9668-a8b11fa3252f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.773627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158feff-ef8c-4962-a8d2-c42dcc042eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.774305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158feff-ef8c-4962-a8d2-c42dcc042eff admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.774821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158feff-ef8c-4962-a8d2-c42dcc042eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.774924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158feff-ef8c-4962-a8d2-c42dcc042eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.775546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158feff-ef8c-4962-a8d2-c42dcc042eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.777387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158feff-ef8c-4962-a8d2-c42dcc042eff admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.788049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1168/2352] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:54.794732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101fee6-b257-4358-b6c5-7ae73bf09408 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.805688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101fee6-b257-4358-b6c5-7ae73bf09408 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.811293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b810e-1568-4080-b785-9fc3bcb52c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.811901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b810e-1568-4080-b785-9fc3bcb52c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.812183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b810e-1568-4080-b785-9fc3bcb52c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.812483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b810e-1568-4080-b785-9fc3bcb52c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.813141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b810e-1568-4080-b785-9fc3bcb52c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.814973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b810e-1568-4080-b785-9fc3bcb52c51 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.827015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1185/2353] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:54.831332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158feff-ef8c-4962-a8d2-c42dcc042eff admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.841480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158feff-ef8c-4962-a8d2-c42dcc042eff admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.846145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.846783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.847059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.847331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] PATH_INFO: `/v3/projects/bbc74a757d3c4e29aa1d1448f659a706/users/49a8558950d34fdba5ab39c70f472cad/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.863251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc74a757d3c4e29aa1d1448f659a706, user_id=49a8558950d34fdba5ab39c70f472cad, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.865950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.950988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1169/2354] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:54 2022] PUT /identity/v3/projects/bbc74a757d3c4e29aa1d1448f659a706/users/49a8558950d34fdba5ab39c70f472cad/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:54.961452 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf3c38-29f1-4e05-90c3-ca36fd409f47 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.961739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf3c38-29f1-4e05-90c3-ca36fd409f47 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.962046 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf3c38-29f1-4e05-90c3-ca36fd409f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.993047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-8ebf3c38-29f1-4e05-90c3-ca36fd409f47 None None] MFA Rules not processed for user `49a8558950d34fdba5ab39c70f472cad`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:54.994357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8ebf3c38-29f1-4e05-90c3-ca36fd409f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:55.072183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1186/2355] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:55:55.083786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:55.086661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:55.181604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464340bc-c80b-4bee-9949-06efb1d094d5 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:55.190340 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f36e1e-d081-4052-944c-4298ef1e6157 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:55.191257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f36e1e-d081-4052-944c-4298ef1e6157 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.191594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f36e1e-d081-4052-944c-4298ef1e6157 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.191868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f36e1e-d081-4052-944c-4298ef1e6157 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.203100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f36e1e-d081-4052-944c-4298ef1e6157 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:55.204712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f36e1e-d081-4052-944c-4298ef1e6157 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:55.214944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1170/2356] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:55:55.448590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf3c38-29f1-4e05-90c3-ca36fd409f47 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:55.516906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf3c38-29f1-4e05-90c3-ca36fd409f47 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:55.522361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:55.523014 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.523291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.523576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] PATH_INFO: `/v3/users/49a8558950d34fdba5ab39c70f472cad` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.524848 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a8558950d34fdba5ab39c70f472cad)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:55.527690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:55.791564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a8558950d34fdba5ab39c70f472cad'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:55.803054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:55.804251 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a8558950d34fdba5ab39c70f472cad'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:55.915841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a8558950d34fdba5ab39c70f472cad'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:55.916474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a8558950d34fdba5ab39c70f472cad'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:55.917338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a8558950d34fdba5ab39c70f472cad'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:55.931149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1187/2357] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:55 2022] DELETE /identity/v3/users/49a8558950d34fdba5ab39c70f472cad => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:55.936299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300697c-0ebe-4c86-87e5-d386768cc6c2 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.936733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300697c-0ebe-4c86-87e5-d386768cc6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.937199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300697c-0ebe-4c86-87e5-d386768cc6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.960133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7300697c-0ebe-4c86-87e5-d386768cc6c2 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:55.961594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7300697c-0ebe-4c86-87e5-d386768cc6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:56.038969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1171/2358] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:56.048816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:56.050396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:56.131956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc2d65-8694-4182-8c9c-82c0298646c3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:56.136439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e218d-90bd-49e6-9539-7f9454d634ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:56.137076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e218d-90bd-49e6-9539-7f9454d634ca service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:56.140283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e218d-90bd-49e6-9539-7f9454d634ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:56.140283 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e218d-90bd-49e6-9539-7f9454d634ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:56.148558 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e218d-90bd-49e6-9539-7f9454d634ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:56.150067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e218d-90bd-49e6-9539-7f9454d634ca service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:56.161606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1188/2359] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:56.663161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300697c-0ebe-4c86-87e5-d386768cc6c2 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:56.664798 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7300697c-0ebe-4c86-87e5-d386768cc6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:56.679642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300697c-0ebe-4c86-87e5-d386768cc6c2 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:56.687165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:56.688016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:56.688418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:56.688800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] PATH_INFO: `/v3/projects/bbc74a757d3c4e29aa1d1448f659a706` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:56.690346 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc74a757d3c4e29aa1d1448f659a706)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:56.693023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:56.836166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:56.854066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc74a757d3c4e29aa1d1448f659a706', 'request_id': 'req-a618c435-4d5d-4124-8f39-274c353bb134'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:56.930337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc74a757d3c4e29aa1d1448f659a706', 'request_id': 'req-a618c435-4d5d-4124-8f39-274c353bb134'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:56.937234 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc74a757d3c4e29aa1d1448f659a706', 'request_id': 'req-a618c435-4d5d-4124-8f39-274c353bb134'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:56.950396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] The token cache is being invalidate because project bbc74a757d3c4e29aa1d1448f659a706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:56.950685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:56.951691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-a618c435-4d5d-4124-8f39-274c353bb134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:56.953969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1172/2360] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:56 2022] DELETE /identity/v3/projects/bbc74a757d3c4e29aa1d1448f659a706 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:56.960858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838735a-5671-4233-961e-f23bad6f27f7 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:56.961534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838735a-5671-4233-961e-f23bad6f27f7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:56.961534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838735a-5671-4233-961e-f23bad6f27f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:56.983551 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7838735a-5671-4233-961e-f23bad6f27f7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:56.985179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7838735a-5671-4233-961e-f23bad6f27f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:57.050048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1189/2361] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:57.058318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f210e6-8422-4264-bca5-f067a9bf602e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.058818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f210e6-8422-4264-bca5-f067a9bf602e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.059038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f210e6-8422-4264-bca5-f067a9bf602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.079110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-80f210e6-8422-4264-bca5-f067a9bf602e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:57.080359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-80f210e6-8422-4264-bca5-f067a9bf602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:57.095786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1173/2362] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:57.100463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838735a-5671-4233-961e-f23bad6f27f7 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:57.111006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838735a-5671-4233-961e-f23bad6f27f7 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:57.116383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:57.117302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.117783 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.118282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.119180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:55:57.119534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:57.121676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:57.134349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1190/2363] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:55:57.139339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f210e6-8422-4264-bca5-f067a9bf602e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:57.149482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f210e6-8422-4264-bca5-f067a9bf602e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:57.157918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f932f-7794-47a3-a77f-5ad30b081f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:57.158550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f932f-7794-47a3-a77f-5ad30b081f9a admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.158836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f932f-7794-47a3-a77f-5ad30b081f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.159111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f932f-7794-47a3-a77f-5ad30b081f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.159753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f932f-7794-47a3-a77f-5ad30b081f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:57.161767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f932f-7794-47a3-a77f-5ad30b081f9a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:57.281015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1174/2364] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:57 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:57.285771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:57.297777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c09306-afb2-439b-a2b7-c25028778325 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:57.357107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da81d6c-6642-4d3c-88e5-1130e25a651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:57.357729 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da81d6c-6642-4d3c-88e5-1130e25a651f admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.358040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da81d6c-6642-4d3c-88e5-1130e25a651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.358528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da81d6c-6642-4d3c-88e5-1130e25a651f admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.358977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da81d6c-6642-4d3c-88e5-1130e25a651f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:57.360901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da81d6c-6642-4d3c-88e5-1130e25a651f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:57.494386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:57.500132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1191/2365] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:57 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:57.506257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f932f-7794-47a3-a77f-5ad30b081f9a admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:57.515601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f932f-7794-47a3-a77f-5ad30b081f9a admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:57.520541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1caa-fd7f-48d6-b3e7-b1619fb7f88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:57.521141 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1caa-fd7f-48d6-b3e7-b1619fb7f88a admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.521421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1caa-fd7f-48d6-b3e7-b1619fb7f88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.521696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1caa-fd7f-48d6-b3e7-b1619fb7f88a admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.522382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a1caa-fd7f-48d6-b3e7-b1619fb7f88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:57.523979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a1caa-fd7f-48d6-b3e7-b1619fb7f88a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:57.536097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1175/2366] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:57.540124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da81d6c-6642-4d3c-88e5-1130e25a651f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:57.551786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da81d6c-6642-4d3c-88e5-1130e25a651f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:57.556587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435b615-5b91-4801-90e0-0bafeb4ce7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:57.558548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b615-5b91-4801-90e0-0bafeb4ce7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.558548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b615-5b91-4801-90e0-0bafeb4ce7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.558548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b615-5b91-4801-90e0-0bafeb4ce7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.559219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435b615-5b91-4801-90e0-0bafeb4ce7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:57.560821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435b615-5b91-4801-90e0-0bafeb4ce7ca admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:57.572608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1192/2367] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:57.579591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1caa-fd7f-48d6-b3e7-b1619fb7f88a admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:57.594749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1caa-fd7f-48d6-b3e7-b1619fb7f88a admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:57.599640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e93a3-edba-4b14-a940-72d1b95d9ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:57.600299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e93a3-edba-4b14-a940-72d1b95d9ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.600579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e93a3-edba-4b14-a940-72d1b95d9ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.600852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e93a3-edba-4b14-a940-72d1b95d9ca3 admin admin] PATH_INFO: `/v3/projects/cba8a2457a504fd88bc674decd1b83c8/users/1acfdbf52b584d56ba1aaeea17b342f7/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.616288 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e93a3-edba-4b14-a940-72d1b95d9ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba8a2457a504fd88bc674decd1b83c8, user_id=1acfdbf52b584d56ba1aaeea17b342f7, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:57.618974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e93a3-edba-4b14-a940-72d1b95d9ca3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:57.719409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1176/2368] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:57 2022] PUT /identity/v3/projects/cba8a2457a504fd88bc674decd1b83c8/users/1acfdbf52b584d56ba1aaeea17b342f7/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:55:57.731772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7ffe0-8a4e-4b1f-b5f0-6bffb87788e7 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.732060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7ffe0-8a4e-4b1f-b5f0-6bffb87788e7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.732338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7ffe0-8a4e-4b1f-b5f0-6bffb87788e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.764403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3aa7ffe0-8a4e-4b1f-b5f0-6bffb87788e7 None None] MFA Rules not processed for user `1acfdbf52b584d56ba1aaeea17b342f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:57.765634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3aa7ffe0-8a4e-4b1f-b5f0-6bffb87788e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:57.840043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1193/2369] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:57.845022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e93a3-edba-4b14-a940-72d1b95d9ca3 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:57.926576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e93a3-edba-4b14-a940-72d1b95d9ca3 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:57.931889 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245045f-5a44-4a99-b234-e189c5347f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:57.940214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245045f-5a44-4a99-b234-e189c5347f7b admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.940214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245045f-5a44-4a99-b234-e189c5347f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.940214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245045f-5a44-4a99-b234-e189c5347f7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.940214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245045f-5a44-4a99-b234-e189c5347f7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:57.940214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245045f-5a44-4a99-b234-e189c5347f7b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.041044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1177/2370] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:57 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:58.045758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7ffe0-8a4e-4b1f-b5f0-6bffb87788e7 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.056904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7ffe0-8a4e-4b1f-b5f0-6bffb87788e7 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.125349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ccab2-dcad-4fd9-b3c0-ab0768b6f196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.126150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ccab2-dcad-4fd9-b3c0-ab0768b6f196 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.126435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ccab2-dcad-4fd9-b3c0-ab0768b6f196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.126714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ccab2-dcad-4fd9-b3c0-ab0768b6f196 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.127356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ccab2-dcad-4fd9-b3c0-ab0768b6f196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.129419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ccab2-dcad-4fd9-b3c0-ab0768b6f196 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.237303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1194/2371] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:58 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:58.241420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245045f-5a44-4a99-b234-e189c5347f7b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.252961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245045f-5a44-4a99-b234-e189c5347f7b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.257702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23591fd6-1a6b-4469-95ac-4556e27450e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.258448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23591fd6-1a6b-4469-95ac-4556e27450e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.258688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23591fd6-1a6b-4469-95ac-4556e27450e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.258946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23591fd6-1a6b-4469-95ac-4556e27450e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.259598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23591fd6-1a6b-4469-95ac-4556e27450e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.261233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23591fd6-1a6b-4469-95ac-4556e27450e8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.272988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1178/2372] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:55:58.277424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ccab2-dcad-4fd9-b3c0-ab0768b6f196 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.288508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ccab2-dcad-4fd9-b3c0-ab0768b6f196 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.297549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea80082-1cba-4413-8761-bc3a3b218e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.298287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea80082-1cba-4413-8761-bc3a3b218e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.298575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea80082-1cba-4413-8761-bc3a3b218e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.298855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea80082-1cba-4413-8761-bc3a3b218e97 admin admin] PATH_INFO: `/v3/projects/be0e1c18117546b0889d165d0eedb0bf/users/fe9028f1415b49e491967fa7d8e76782/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.316863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea80082-1cba-4413-8761-bc3a3b218e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0e1c18117546b0889d165d0eedb0bf, user_id=fe9028f1415b49e491967fa7d8e76782, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.319594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea80082-1cba-4413-8761-bc3a3b218e97 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.420317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1195/2373] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:58 2022] PUT /identity/v3/projects/be0e1c18117546b0889d165d0eedb0bf/users/fe9028f1415b49e491967fa7d8e76782/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:58.425169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23591fd6-1a6b-4469-95ac-4556e27450e8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.438105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23591fd6-1a6b-4469-95ac-4556e27450e8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.466562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:58.515786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4ab1c-ff92-4c23-b157-eba80f7fd084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.516501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4ab1c-ff92-4c23-b157-eba80f7fd084 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.516880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4ab1c-ff92-4c23-b157-eba80f7fd084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.517170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4ab1c-ff92-4c23-b157-eba80f7fd084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.518220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4ab1c-ff92-4c23-b157-eba80f7fd084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.519929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4ab1c-ff92-4c23-b157-eba80f7fd084 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.548620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1179/2374] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:58.550351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea80082-1cba-4413-8761-bc3a3b218e97 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.567799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea80082-1cba-4413-8761-bc3a3b218e97 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.573539 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.575437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.575959 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.579170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] PATH_INFO: `/v3/projects/be0e1c18117546b0889d165d0eedb0bf/users/fe9028f1415b49e491967fa7d8e76782/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.580711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0e1c18117546b0889d165d0eedb0bf, user_id=fe9028f1415b49e491967fa7d8e76782, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.586027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.601191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1196/2375] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:58 2022] PUT /identity/v3/projects/be0e1c18117546b0889d165d0eedb0bf/users/fe9028f1415b49e491967fa7d8e76782/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:58.614863 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75246a-79e6-4210-b1cc-99722a083346 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.615352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75246a-79e6-4210-b1cc-99722a083346 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.615714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75246a-79e6-4210-b1cc-99722a083346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.654497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0b75246a-79e6-4210-b1cc-99722a083346 None None] MFA Rules not processed for user `fe9028f1415b49e491967fa7d8e76782`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:58.657559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0b75246a-79e6-4210-b1cc-99722a083346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:58.742992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1180/2376] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:58.756044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.757882 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:58.797494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:58.824953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e875c3-bf5c-4548-bfd6-72951f8c1d46 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.828912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370c506-8aaa-49b8-a33b-dd45ff1ec61c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.829547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c506-8aaa-49b8-a33b-dd45ff1ec61c service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.829823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c506-8aaa-49b8-a33b-dd45ff1ec61c service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.830119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c506-8aaa-49b8-a33b-dd45ff1ec61c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.844282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370c506-8aaa-49b8-a33b-dd45ff1ec61c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.845785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370c506-8aaa-49b8-a33b-dd45ff1ec61c service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.861484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1197/2377] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:00.138432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75246a-79e6-4210-b1cc-99722a083346 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.149092 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75246a-79e6-4210-b1cc-99722a083346 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.154597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe9028f1415b49e491967fa7d8e76782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be0e1c18117546b0889d165d0eedb0bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.155221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.155501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.155768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.156619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:00.156791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.158383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.175295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1181/2378] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:56:00.176654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370c506-8aaa-49b8-a33b-dd45ff1ec61c service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.186472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370c506-8aaa-49b8-a33b-dd45ff1ec61c service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.191297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437132-6d19-4db5-82bb-088982c243dd tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe9028f1415b49e491967fa7d8e76782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be0e1c18117546b0889d165d0eedb0bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.191915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437132-6d19-4db5-82bb-088982c243dd tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.192199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437132-6d19-4db5-82bb-088982c243dd tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.192472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437132-6d19-4db5-82bb-088982c243dd tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.193118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437132-6d19-4db5-82bb-088982c243dd tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.195087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437132-6d19-4db5-82bb-088982c243dd tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.305458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1198/2379] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:00 2022] POST /identity/v3/projects => generated 352 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:00.310142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.322409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242017-a68a-4098-80db-01cfc0620d55 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.412067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe9028f1415b49e491967fa7d8e76782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be0e1c18117546b0889d165d0eedb0bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.412810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.413316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.413697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.414477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:00.414887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.417373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.431569 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1182/2380] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:56:00.476722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437132-6d19-4db5-82bb-088982c243dd tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.487624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437132-6d19-4db5-82bb-088982c243dd tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.493435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe9028f1415b49e491967fa7d8e76782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be0e1c18117546b0889d165d0eedb0bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.494678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.495191 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.495676 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] PATH_INFO: `/v3/projects/d55e2f7fd67c49c184504e9cfd7b1a3c` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.499224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d55e2f7fd67c49c184504e9cfd7b1a3c)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.501251 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.657705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55e2f7fd67c49c184504e9cfd7b1a3c', 'request_id': 'req-f1854945-482f-4c0a-8187-a39af645aafa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:00.767872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55e2f7fd67c49c184504e9cfd7b1a3c', 'request_id': 'req-f1854945-482f-4c0a-8187-a39af645aafa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:00.775700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55e2f7fd67c49c184504e9cfd7b1a3c', 'request_id': 'req-f1854945-482f-4c0a-8187-a39af645aafa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:00.787285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] The token cache is being invalidate because project d55e2f7fd67c49c184504e9cfd7b1a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:00.787574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:00.788281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f1854945-482f-4c0a-8187-a39af645aafa tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:00.789799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1199/2381] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:00 2022] DELETE /identity/v3/projects/d55e2f7fd67c49c184504e9cfd7b1a3c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:01.655952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:01.658055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:01.691969 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:01.726771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c93dfb-76a3-4c6d-b806-39c961494f81 tempest-VolumeQuotaClassesTest-1284171239 tempest-VolumeQuotaClassesTest-1284171239-project-admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:01.790741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:01.791939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:01.792222 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:01.792541 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] PATH_INFO: `/v3/users/1acfdbf52b584d56ba1aaeea17b342f7` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:01.793787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1acfdbf52b584d56ba1aaeea17b342f7)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:01.795809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:02.024440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1acfdbf52b584d56ba1aaeea17b342f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:02.105125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1acfdbf52b584d56ba1aaeea17b342f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:02.106283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1acfdbf52b584d56ba1aaeea17b342f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:02.116589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1acfdbf52b584d56ba1aaeea17b342f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:02.117536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1acfdbf52b584d56ba1aaeea17b342f7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:02.138695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1183/2382] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:01 2022] DELETE /identity/v3/users/1acfdbf52b584d56ba1aaeea17b342f7 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:02.144360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59be145-796a-4bed-8621-46d32897a382 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:02.146570 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59be145-796a-4bed-8621-46d32897a382 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:02.150600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59be145-796a-4bed-8621-46d32897a382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:02.166425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-e59be145-796a-4bed-8621-46d32897a382 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:02.167711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e59be145-796a-4bed-8621-46d32897a382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:02.231545 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1200/2383] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:02.241527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:02.243069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:02.311722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272955e-40ae-493b-a3c1-80b70c06e804 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:02.315402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370c228-25b5-43c1-aa88-5b74c9d05eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:02.316011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370c228-25b5-43c1-aa88-5b74c9d05eaf service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:02.316300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370c228-25b5-43c1-aa88-5b74c9d05eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:02.316572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370c228-25b5-43c1-aa88-5b74c9d05eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:02.326704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370c228-25b5-43c1-aa88-5b74c9d05eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:02.328138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370c228-25b5-43c1-aa88-5b74c9d05eaf service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:02.332453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:02.339048 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1184/2384] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:02.943772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59be145-796a-4bed-8621-46d32897a382 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:02.946365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e59be145-796a-4bed-8621-46d32897a382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:02.962527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59be145-796a-4bed-8621-46d32897a382 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:02.969622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:02.970488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:02.970942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:02.971314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] PATH_INFO: `/v3/projects/cba8a2457a504fd88bc674decd1b83c8` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:02.972699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba8a2457a504fd88bc674decd1b83c8)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:02.976810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:03.083088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:03.097181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba8a2457a504fd88bc674decd1b83c8', 'request_id': 'req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.179362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba8a2457a504fd88bc674decd1b83c8', 'request_id': 'req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.186655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba8a2457a504fd88bc674decd1b83c8', 'request_id': 'req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.197835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] The token cache is being invalidate because project cba8a2457a504fd88bc674decd1b83c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:03.198179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.200017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.201568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1201/2385] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:02 2022] DELETE /identity/v3/projects/cba8a2457a504fd88bc674decd1b83c8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:03.206081 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370c228-25b5-43c1-aa88-5b74c9d05eaf service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:03.217720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370c228-25b5-43c1-aa88-5b74c9d05eaf service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:03.275593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:03.276244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:03.276528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:03.276802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] PATH_INFO: `/v3/users/fe9028f1415b49e491967fa7d8e76782` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:03.278595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9028f1415b49e491967fa7d8e76782)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:03.280697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:03.535539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9028f1415b49e491967fa7d8e76782'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.621877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9028f1415b49e491967fa7d8e76782'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.622563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9028f1415b49e491967fa7d8e76782'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.634030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9028f1415b49e491967fa7d8e76782'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.635182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec4e2b75-b95e-45b2-84b5-a09faa591f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9028f1415b49e491967fa7d8e76782'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:03.655516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1185/2386] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:03 2022] DELETE /identity/v3/users/fe9028f1415b49e491967fa7d8e76782 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:04.242662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:04.244224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:04.259372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762c49c-59b5-4fbf-8ba0-2c686aea3d6f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:04.333617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:04.334456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:04.334770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:04.335042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] PATH_INFO: `/v3/projects/be0e1c18117546b0889d165d0eedb0bf` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:04.336330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0e1c18117546b0889d165d0eedb0bf)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:04.338357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:04.456453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0e1c18117546b0889d165d0eedb0bf', 'request_id': 'req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.552690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0e1c18117546b0889d165d0eedb0bf', 'request_id': 'req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.560513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0e1c18117546b0889d165d0eedb0bf', 'request_id': 'req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.573247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] The token cache is being invalidate because project be0e1c18117546b0889d165d0eedb0bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:04.573540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.574433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-78d7c6aa-61a9-4cd9-b2e2-2d430450d6b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.576133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1202/2387] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:04 2022] DELETE /identity/v3/projects/be0e1c18117546b0889d165d0eedb0bf => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:04.581964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542162d-417b-46c8-bde9-5e6a725d5328 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:04.582648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542162d-417b-46c8-bde9-5e6a725d5328 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:04.582648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542162d-417b-46c8-bde9-5e6a725d5328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:04.603627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-4542162d-417b-46c8-bde9-5e6a725d5328 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:04.604856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4542162d-417b-46c8-bde9-5e6a725d5328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:04.692700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1186/2388] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:04.706176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38faf4-58a2-4903-a316-b477dc5e096b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:04.706706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38faf4-58a2-4903-a316-b477dc5e096b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:04.706987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38faf4-58a2-4903-a316-b477dc5e096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:04.731365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-8d38faf4-58a2-4903-a316-b477dc5e096b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:04.732646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8d38faf4-58a2-4903-a316-b477dc5e096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:04.744497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1203/2389] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:04.747721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542162d-417b-46c8-bde9-5e6a725d5328 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:04.769464 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542162d-417b-46c8-bde9-5e6a725d5328 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:04.774733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:04.775309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:04.776184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:04.776184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:04.776583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:04.776807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:04.778466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:04.791816 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1187/2390] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:04.796740 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38faf4-58a2-4903-a316-b477dc5e096b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:04.806358 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38faf4-58a2-4903-a316-b477dc5e096b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:04.814469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e5765-f534-40e1-a7b4-6ebf6ac1d29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:04.815268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e5765-f534-40e1-a7b4-6ebf6ac1d29e admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:04.815451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e5765-f534-40e1-a7b4-6ebf6ac1d29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:04.815694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e5765-f534-40e1-a7b4-6ebf6ac1d29e admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:04.816322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e5765-f534-40e1-a7b4-6ebf6ac1d29e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:04.818247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e5765-f534-40e1-a7b4-6ebf6ac1d29e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:04.922264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1204/2391] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:04 2022] POST /identity/v3/projects => generated 401 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:04.923554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:04.930564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:04.935742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e67b1-17f1-4eaf-910a-5e84268c89a6 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.002852 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3e70c-e056-42fc-88ef-1ed792d13662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.003469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3e70c-e056-42fc-88ef-1ed792d13662 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.003754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3e70c-e056-42fc-88ef-1ed792d13662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.004034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3e70c-e056-42fc-88ef-1ed792d13662 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.004666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3e70c-e056-42fc-88ef-1ed792d13662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.006870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3e70c-e056-42fc-88ef-1ed792d13662 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.091068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1188/2392] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:04 2022] POST /identity/v3/users => generated 301 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:05.095425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e5765-f534-40e1-a7b4-6ebf6ac1d29e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.107025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e5765-f534-40e1-a7b4-6ebf6ac1d29e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.112905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f3b0c-a69c-4e26-9d00-299d55ba2289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.113107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f3b0c-a69c-4e26-9d00-299d55ba2289 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.113357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f3b0c-a69c-4e26-9d00-299d55ba2289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.113635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f3b0c-a69c-4e26-9d00-299d55ba2289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.114335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f3b0c-a69c-4e26-9d00-299d55ba2289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.115950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f3b0c-a69c-4e26-9d00-299d55ba2289 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.131464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1205/2393] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:05.132967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3e70c-e056-42fc-88ef-1ed792d13662 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.146501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3e70c-e056-42fc-88ef-1ed792d13662 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.151775 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16662313-36a1-4bb6-9cef-673894d3b842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.153232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16662313-36a1-4bb6-9cef-673894d3b842 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.153685 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16662313-36a1-4bb6-9cef-673894d3b842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.154032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16662313-36a1-4bb6-9cef-673894d3b842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.154879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16662313-36a1-4bb6-9cef-673894d3b842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.156865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16662313-36a1-4bb6-9cef-673894d3b842 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.176927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1189/2394] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:05.178291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f3b0c-a69c-4e26-9d00-299d55ba2289 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.189781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f3b0c-a69c-4e26-9d00-299d55ba2289 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.194583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553baa4-52ba-467c-9685-4adc5fd292bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.195313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553baa4-52ba-467c-9685-4adc5fd292bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.195655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553baa4-52ba-467c-9685-4adc5fd292bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.195992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553baa4-52ba-467c-9685-4adc5fd292bc admin admin] PATH_INFO: `/v3/projects/b037947f03b449529a271cb68e73a3ba/users/c603d4d72e524ba68ab1f6bc5b5945c5/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.200480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:05.212416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553baa4-52ba-467c-9685-4adc5fd292bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b037947f03b449529a271cb68e73a3ba, user_id=c603d4d72e524ba68ab1f6bc5b5945c5, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.215867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553baa4-52ba-467c-9685-4adc5fd292bc admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.331130 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1206/2395] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:05 2022] PUT /identity/v3/projects/b037947f03b449529a271cb68e73a3ba/users/c603d4d72e524ba68ab1f6bc5b5945c5/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:05.343337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a978835-f7ae-4f1b-8be9-9e65e12d7292 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.343619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a978835-f7ae-4f1b-8be9-9e65e12d7292 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.343898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a978835-f7ae-4f1b-8be9-9e65e12d7292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.376751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7a978835-f7ae-4f1b-8be9-9e65e12d7292 None None] MFA Rules not processed for user `c603d4d72e524ba68ab1f6bc5b5945c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:05.378035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7a978835-f7ae-4f1b-8be9-9e65e12d7292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:05.461136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1190/2396] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:05 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:05.467203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553baa4-52ba-467c-9685-4adc5fd292bc admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.559821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553baa4-52ba-467c-9685-4adc5fd292bc admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.564972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5bbab-d508-4437-ad48-a87274286529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.565649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5bbab-d508-4437-ad48-a87274286529 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.565984 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5bbab-d508-4437-ad48-a87274286529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.566266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5bbab-d508-4437-ad48-a87274286529 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.566943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5bbab-d508-4437-ad48-a87274286529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.568911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5bbab-d508-4437-ad48-a87274286529 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.671530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1207/2397] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:05 2022] POST /identity/v3/projects => generated 401 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:05.675896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a978835-f7ae-4f1b-8be9-9e65e12d7292 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.691931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a978835-f7ae-4f1b-8be9-9e65e12d7292 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.785090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb8e90-4985-4eb2-a788-e4b75f78a755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.785746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb8e90-4985-4eb2-a788-e4b75f78a755 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.786198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb8e90-4985-4eb2-a788-e4b75f78a755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.786317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb8e90-4985-4eb2-a788-e4b75f78a755 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.787011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb8e90-4985-4eb2-a788-e4b75f78a755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.788873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb8e90-4985-4eb2-a788-e4b75f78a755 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.910118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1191/2398] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:05 2022] POST /identity/v3/users => generated 307 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:05.917041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5bbab-d508-4437-ad48-a87274286529 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.932759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5bbab-d508-4437-ad48-a87274286529 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.941088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19e279-79b6-4084-bb52-6be1ed524c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.948624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19e279-79b6-4084-bb52-6be1ed524c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.948624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19e279-79b6-4084-bb52-6be1ed524c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.948624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19e279-79b6-4084-bb52-6be1ed524c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.948624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19e279-79b6-4084-bb52-6be1ed524c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.951395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19e279-79b6-4084-bb52-6be1ed524c94 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.981911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1208/2399] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:05.983216 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb8e90-4985-4eb2-a788-e4b75f78a755 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.992520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:06.005057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb8e90-4985-4eb2-a788-e4b75f78a755 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.010320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e081399-a02d-46ae-82bb-15ee21fb3df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.011160 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081399-a02d-46ae-82bb-15ee21fb3df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.012131 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081399-a02d-46ae-82bb-15ee21fb3df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.012983 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081399-a02d-46ae-82bb-15ee21fb3df4 admin admin] PATH_INFO: `/v3/projects/3d13da556eca42bfb8f2a2db76ff9819/users/a259c62a5e4248b9947603ac8c56262e/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.033385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e081399-a02d-46ae-82bb-15ee21fb3df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d13da556eca42bfb8f2a2db76ff9819, user_id=a259c62a5e4248b9947603ac8c56262e, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.036183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e081399-a02d-46ae-82bb-15ee21fb3df4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.137716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1192/2400] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:05 2022] PUT /identity/v3/projects/3d13da556eca42bfb8f2a2db76ff9819/users/a259c62a5e4248b9947603ac8c56262e/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:06.139764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19e279-79b6-4084-bb52-6be1ed524c94 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.153540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19e279-79b6-4084-bb52-6be1ed524c94 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.217315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192573ba-c783-4c5e-899f-499318f5faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.217968 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192573ba-c783-4c5e-899f-499318f5faac admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.218256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192573ba-c783-4c5e-899f-499318f5faac admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.218524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192573ba-c783-4c5e-899f-499318f5faac admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.219177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192573ba-c783-4c5e-899f-499318f5faac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.220784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192573ba-c783-4c5e-899f-499318f5faac admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.232660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1209/2401] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:06.237088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e081399-a02d-46ae-82bb-15ee21fb3df4 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.248475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e081399-a02d-46ae-82bb-15ee21fb3df4 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.253163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.253820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.254148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.254423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] PATH_INFO: `/v3/projects/3d13da556eca42bfb8f2a2db76ff9819/users/a259c62a5e4248b9947603ac8c56262e/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.256370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d13da556eca42bfb8f2a2db76ff9819, user_id=a259c62a5e4248b9947603ac8c56262e, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.259171 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.270815 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1193/2402] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:06 2022] PUT /identity/v3/projects/3d13da556eca42bfb8f2a2db76ff9819/users/a259c62a5e4248b9947603ac8c56262e/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:06.282507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a8f15-83e1-49fb-ab7e-7a71daae4a9e None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.282994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a8f15-83e1-49fb-ab7e-7a71daae4a9e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.283276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a8f15-83e1-49fb-ab7e-7a71daae4a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.319386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3e4a8f15-83e1-49fb-ab7e-7a71daae4a9e None None] MFA Rules not processed for user `a259c62a5e4248b9947603ac8c56262e`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:06.320702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3e4a8f15-83e1-49fb-ab7e-7a71daae4a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:06.375263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:06.403404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1210/2403] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:06 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:06.414416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.416509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:06.498929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b89d-794f-4f60-b96c-bd38c028b8cd admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.503365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023da687-1412-4ac8-b6c8-d2dc630df390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.504253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da687-1412-4ac8-b6c8-d2dc630df390 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.504601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da687-1412-4ac8-b6c8-d2dc630df390 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.504975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da687-1412-4ac8-b6c8-d2dc630df390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.515661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023da687-1412-4ac8-b6c8-d2dc630df390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.517555 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023da687-1412-4ac8-b6c8-d2dc630df390 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.532908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1194/2404] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:06 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:06.865765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a8f15-83e1-49fb-ab7e-7a71daae4a9e None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.891161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a8f15-83e1-49fb-ab7e-7a71daae4a9e None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.895309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d0de5-97c8-44d5-ac04-c2b9224eead8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.895926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0de5-97c8-44d5-ac04-c2b9224eead8 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.896347 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0de5-97c8-44d5-ac04-c2b9224eead8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.896629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0de5-97c8-44d5-ac04-c2b9224eead8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.003166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d0de5-97c8-44d5-ac04-c2b9224eead8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.004804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d0de5-97c8-44d5-ac04-c2b9224eead8 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.037152 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1211/2405] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:06 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:12.915373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023da687-1412-4ac8-b6c8-d2dc630df390 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:13.001722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023da687-1412-4ac8-b6c8-d2dc630df390 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:13.006820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:13.008702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:13.008702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:13.008702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] PATH_INFO: `/v3/users/c603d4d72e524ba68ab1f6bc5b5945c5` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:13.009514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c603d4d72e524ba68ab1f6bc5b5945c5)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:13.011742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:13.223715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c603d4d72e524ba68ab1f6bc5b5945c5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:13.312501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c603d4d72e524ba68ab1f6bc5b5945c5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:13.313208 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c603d4d72e524ba68ab1f6bc5b5945c5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:13.326069 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c603d4d72e524ba68ab1f6bc5b5945c5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:13.327409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c603d4d72e524ba68ab1f6bc5b5945c5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:13.333829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:13.349630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1195/2406] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:12 2022] DELETE /identity/v3/users/c603d4d72e524ba68ab1f6bc5b5945c5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:13.355974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaab51-4008-476b-8f59-ae565a186842 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:13.357116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaab51-4008-476b-8f59-ae565a186842 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:13.357893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaab51-4008-476b-8f59-ae565a186842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:13.380956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a5eaab51-4008-476b-8f59-ae565a186842 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:13.382441 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a5eaab51-4008-476b-8f59-ae565a186842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:13.493467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1212/2407] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:13.507045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:13.510513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:13.623472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a869875-545a-4ac3-9f1f-464e9493b525 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:13.629946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efa5d9-83b2-409d-a7d2-0305ebbf47e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:13.633896 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efa5d9-83b2-409d-a7d2-0305ebbf47e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:13.635274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efa5d9-83b2-409d-a7d2-0305ebbf47e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:13.635649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efa5d9-83b2-409d-a7d2-0305ebbf47e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:13.659598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efa5d9-83b2-409d-a7d2-0305ebbf47e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:13.662824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efa5d9-83b2-409d-a7d2-0305ebbf47e1 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:13.678122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1196/2408] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:14.452518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaab51-4008-476b-8f59-ae565a186842 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:14.454643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a5eaab51-4008-476b-8f59-ae565a186842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:14.468189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaab51-4008-476b-8f59-ae565a186842 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:14.474728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:14.475597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:14.476042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:14.476557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] PATH_INFO: `/v3/projects/b037947f03b449529a271cb68e73a3ba` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:14.477914 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b037947f03b449529a271cb68e73a3ba)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:14.480221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:14.687239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b037947f03b449529a271cb68e73a3ba', 'request_id': 'req-f96af8f1-f6c0-4278-a97d-dc21015823b5'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.862313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b037947f03b449529a271cb68e73a3ba', 'request_id': 'req-f96af8f1-f6c0-4278-a97d-dc21015823b5'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.881049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b037947f03b449529a271cb68e73a3ba', 'request_id': 'req-f96af8f1-f6c0-4278-a97d-dc21015823b5'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.885690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] The token cache is being invalidate because project b037947f03b449529a271cb68e73a3ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:14.886245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.887050 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.890153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1213/2409] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:14 2022] DELETE /identity/v3/projects/b037947f03b449529a271cb68e73a3ba => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:14.896363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efa5d9-83b2-409d-a7d2-0305ebbf47e1 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:14.924156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efa5d9-83b2-409d-a7d2-0305ebbf47e1 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:15.009097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:15.009895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:15.010196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:15.010473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] PATH_INFO: `/v3/users/a259c62a5e4248b9947603ac8c56262e` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:15.012493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a259c62a5e4248b9947603ac8c56262e)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:15.014742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:15.283014 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a259c62a5e4248b9947603ac8c56262e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.373781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a259c62a5e4248b9947603ac8c56262e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.374781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a259c62a5e4248b9947603ac8c56262e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.385143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a259c62a5e4248b9947603ac8c56262e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.386004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-f3ca2704-a90c-43f9-bbd3-71267b42d879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a259c62a5e4248b9947603ac8c56262e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.411257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1197/2410] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:14 2022] DELETE /identity/v3/users/a259c62a5e4248b9947603ac8c56262e => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:15.956647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:15.958310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:15.968668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:15.974515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96af8f1-f6c0-4278-a97d-dc21015823b5 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.048177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.048867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.049385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.049890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] PATH_INFO: `/v3/projects/3d13da556eca42bfb8f2a2db76ff9819` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.051226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d13da556eca42bfb8f2a2db76ff9819)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.053169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.166359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d13da556eca42bfb8f2a2db76ff9819', 'request_id': 'req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.278498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d13da556eca42bfb8f2a2db76ff9819', 'request_id': 'req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.290215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d13da556eca42bfb8f2a2db76ff9819', 'request_id': 'req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.314638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] The token cache is being invalidate because project 3d13da556eca42bfb8f2a2db76ff9819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:16.315055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.317547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-0342543d-e3e0-4894-8512-5e3e9fc0a8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.320382 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1214/2411] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:15 2022] DELETE /identity/v3/projects/3d13da556eca42bfb8f2a2db76ff9819 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:16.326175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d23a6-5f1c-4c5c-ba7d-abef0f2a833e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.328447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d23a6-5f1c-4c5c-ba7d-abef0f2a833e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.328957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d23a6-5f1c-4c5c-ba7d-abef0f2a833e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.351559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-8f2d23a6-5f1c-4c5c-ba7d-abef0f2a833e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:16.352790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8f2d23a6-5f1c-4c5c-ba7d-abef0f2a833e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:16.357882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:16.424968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1198/2412] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:16.433760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6082d9-bcb5-4cb3-99ed-41b77e6f2a07 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.434094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6082d9-bcb5-4cb3-99ed-41b77e6f2a07 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.434372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6082d9-bcb5-4cb3-99ed-41b77e6f2a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.455487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4c6082d9-bcb5-4cb3-99ed-41b77e6f2a07 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:16.456814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4c6082d9-bcb5-4cb3-99ed-41b77e6f2a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:16.474391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1215/2413] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:16.476205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d23a6-5f1c-4c5c-ba7d-abef0f2a833e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.490449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d23a6-5f1c-4c5c-ba7d-abef0f2a833e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.495924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.496661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.496954 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.497262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.497996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:16.498315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.499961 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.523112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1199/2414] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:16.524989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6082d9-bcb5-4cb3-99ed-41b77e6f2a07 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.536324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6082d9-bcb5-4cb3-99ed-41b77e6f2a07 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.541327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76859a25-0aea-4fa7-a7df-b431d29773a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.542310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859a25-0aea-4fa7-a7df-b431d29773a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.542464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859a25-0aea-4fa7-a7df-b431d29773a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.542731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859a25-0aea-4fa7-a7df-b431d29773a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.543378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76859a25-0aea-4fa7-a7df-b431d29773a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.545300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76859a25-0aea-4fa7-a7df-b431d29773a0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.666124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1216/2415] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:16.670477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.682120 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5d64e-f682-4a0e-8a1d-d9f238152848 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.747311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ba39-9311-4523-a15a-0a642ebcf803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.747920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ba39-9311-4523-a15a-0a642ebcf803 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.748200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ba39-9311-4523-a15a-0a642ebcf803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.748474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ba39-9311-4523-a15a-0a642ebcf803 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.749104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3ba39-9311-4523-a15a-0a642ebcf803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.751101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3ba39-9311-4523-a15a-0a642ebcf803 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.858366 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1200/2416] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:16.861073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76859a25-0aea-4fa7-a7df-b431d29773a0 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.872869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76859a25-0aea-4fa7-a7df-b431d29773a0 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.879187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cd22a-c5af-4287-a756-ffdd2b277f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.884964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cd22a-c5af-4287-a756-ffdd2b277f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.884964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cd22a-c5af-4287-a756-ffdd2b277f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.884964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cd22a-c5af-4287-a756-ffdd2b277f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.884964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cd22a-c5af-4287-a756-ffdd2b277f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.884964 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cd22a-c5af-4287-a756-ffdd2b277f3d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.901170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1217/2417] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:16.906361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ba39-9311-4523-a15a-0a642ebcf803 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.918855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ba39-9311-4523-a15a-0a642ebcf803 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.924125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ca868-4fb6-43b8-98ff-1b208d77bf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.924736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ca868-4fb6-43b8-98ff-1b208d77bf0a admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.925022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ca868-4fb6-43b8-98ff-1b208d77bf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.925298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ca868-4fb6-43b8-98ff-1b208d77bf0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.926059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ca868-4fb6-43b8-98ff-1b208d77bf0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.927809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ca868-4fb6-43b8-98ff-1b208d77bf0a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.940584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1201/2418] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:16.945028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cd22a-c5af-4287-a756-ffdd2b277f3d admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.958516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cd22a-c5af-4287-a756-ffdd2b277f3d admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.973900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9868ab0-b730-4178-affa-8c3292259063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.973900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9868ab0-b730-4178-affa-8c3292259063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.973900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9868ab0-b730-4178-affa-8c3292259063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.973900 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9868ab0-b730-4178-affa-8c3292259063 admin admin] PATH_INFO: `/v3/projects/58cd72b7c46f42fe8b69aed3982fa8c8/users/1e5b270b2ac14903a55936e1d5069d65/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.001887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9868ab0-b730-4178-affa-8c3292259063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cd72b7c46f42fe8b69aed3982fa8c8, user_id=1e5b270b2ac14903a55936e1d5069d65, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:17.005167 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9868ab0-b730-4178-affa-8c3292259063 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:17.096073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1218/2419] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:16 2022] PUT /identity/v3/projects/58cd72b7c46f42fe8b69aed3982fa8c8/users/1e5b270b2ac14903a55936e1d5069d65/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:17.108667 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b733f7-2138-42f7-b43d-aa5405fb3696 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:17.109028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b733f7-2138-42f7-b43d-aa5405fb3696 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:17.109403 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b733f7-2138-42f7-b43d-aa5405fb3696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.140576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f5b733f7-2138-42f7-b43d-aa5405fb3696 None None] MFA Rules not processed for user `1e5b270b2ac14903a55936e1d5069d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:17.141834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f5b733f7-2138-42f7-b43d-aa5405fb3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:17.212725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1202/2420] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:17.217874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9868ab0-b730-4178-affa-8c3292259063 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:17.257426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:17.296706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9868ab0-b730-4178-affa-8c3292259063 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:17.301474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f73ad-932c-4b6a-ba31-b15e67ea9912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:17.302113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f73ad-932c-4b6a-ba31-b15e67ea9912 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:17.302396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f73ad-932c-4b6a-ba31-b15e67ea9912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:17.302672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f73ad-932c-4b6a-ba31-b15e67ea9912 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.303303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f73ad-932c-4b6a-ba31-b15e67ea9912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:17.305285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f73ad-932c-4b6a-ba31-b15e67ea9912 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:17.416269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1219/2421] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:17 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:17.419865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b733f7-2138-42f7-b43d-aa5405fb3696 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:17.427067 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:17.431354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b733f7-2138-42f7-b43d-aa5405fb3696 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:17.493167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afeb06-ceee-483b-b2a1-9d1593e48381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:17.493697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afeb06-ceee-483b-b2a1-9d1593e48381 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:17.494023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afeb06-ceee-483b-b2a1-9d1593e48381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:17.494303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afeb06-ceee-483b-b2a1-9d1593e48381 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.494938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afeb06-ceee-483b-b2a1-9d1593e48381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:17.497451 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afeb06-ceee-483b-b2a1-9d1593e48381 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:17.746715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1203/2422] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:17 2022] POST /identity/v3/users => generated 298 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:17.750783 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f73ad-932c-4b6a-ba31-b15e67ea9912 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:17.762295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f73ad-932c-4b6a-ba31-b15e67ea9912 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:17.767272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49108212-9773-40e6-8f70-2b55c0bc81fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:17.767912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49108212-9773-40e6-8f70-2b55c0bc81fa admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:17.768202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49108212-9773-40e6-8f70-2b55c0bc81fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:17.768468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49108212-9773-40e6-8f70-2b55c0bc81fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.769117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49108212-9773-40e6-8f70-2b55c0bc81fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:17.770803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49108212-9773-40e6-8f70-2b55c0bc81fa admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:17.781243 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1220/2423] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:17.785612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afeb06-ceee-483b-b2a1-9d1593e48381 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:17.797027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afeb06-ceee-483b-b2a1-9d1593e48381 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:17.801982 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21933c05-4f7c-44f2-adbf-6b3b19a3a611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:17.802655 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21933c05-4f7c-44f2-adbf-6b3b19a3a611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:17.802984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21933c05-4f7c-44f2-adbf-6b3b19a3a611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:17.803197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21933c05-4f7c-44f2-adbf-6b3b19a3a611 admin admin] PATH_INFO: `/v3/projects/42723667b25c4a1aa60ae1c6a0aeba11/users/e8f7a14f4ca04d6f96d0e54111c773f3/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.820705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21933c05-4f7c-44f2-adbf-6b3b19a3a611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42723667b25c4a1aa60ae1c6a0aeba11, user_id=e8f7a14f4ca04d6f96d0e54111c773f3, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:17.823060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21933c05-4f7c-44f2-adbf-6b3b19a3a611 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:17.937429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1204/2424] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:17 2022] PUT /identity/v3/projects/42723667b25c4a1aa60ae1c6a0aeba11/users/e8f7a14f4ca04d6f96d0e54111c773f3/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:17.945970 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49108212-9773-40e6-8f70-2b55c0bc81fa admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:17.963981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49108212-9773-40e6-8f70-2b55c0bc81fa admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:18.038920 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27446c90-433e-4178-b000-eb046e11342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:18.039543 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446c90-433e-4178-b000-eb046e11342f admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:18.039824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446c90-433e-4178-b000-eb046e11342f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:18.040103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446c90-433e-4178-b000-eb046e11342f admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:18.043662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27446c90-433e-4178-b000-eb046e11342f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:18.043662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27446c90-433e-4178-b000-eb046e11342f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:18.054945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1221/2425] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:18.059443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21933c05-4f7c-44f2-adbf-6b3b19a3a611 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:18.081902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21933c05-4f7c-44f2-adbf-6b3b19a3a611 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:18.097556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:18.097556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:18.097556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:18.097556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] PATH_INFO: `/v3/projects/42723667b25c4a1aa60ae1c6a0aeba11/users/e8f7a14f4ca04d6f96d0e54111c773f3/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:18.100691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42723667b25c4a1aa60ae1c6a0aeba11, user_id=e8f7a14f4ca04d6f96d0e54111c773f3, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:18.106429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:18.117984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1205/2426] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:18 2022] PUT /identity/v3/projects/42723667b25c4a1aa60ae1c6a0aeba11/users/e8f7a14f4ca04d6f96d0e54111c773f3/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:18.132722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec3495-1079-4a1a-b35b-dcebb7aba751 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:18.132722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec3495-1079-4a1a-b35b-dcebb7aba751 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:18.132722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec3495-1079-4a1a-b35b-dcebb7aba751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:18.163928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-9fec3495-1079-4a1a-b35b-dcebb7aba751 None None] MFA Rules not processed for user `e8f7a14f4ca04d6f96d0e54111c773f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:18.165299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9fec3495-1079-4a1a-b35b-dcebb7aba751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:18.240227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1222/2427] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:18.251546 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:18.253219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:18.314032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582acda-10e0-4d64-a4c0-0e4dabd78272 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:18.317762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3dcd2-f79d-48e4-8590-dc703fe3735e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:18.318399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3dcd2-f79d-48e4-8590-dc703fe3735e service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:18.318683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3dcd2-f79d-48e4-8590-dc703fe3735e service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:18.318985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3dcd2-f79d-48e4-8590-dc703fe3735e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:18.329765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3dcd2-f79d-48e4-8590-dc703fe3735e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:18.336088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3dcd2-f79d-48e4-8590-dc703fe3735e service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:18.342364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1206/2428] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:19.563898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec3495-1079-4a1a-b35b-dcebb7aba751 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:19.596718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:19.635918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec3495-1079-4a1a-b35b-dcebb7aba751 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:19.641335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:19.642469 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:19.642751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:19.643024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] PATH_INFO: `/v3/users/1e5b270b2ac14903a55936e1d5069d65` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:19.644249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5b270b2ac14903a55936e1d5069d65)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:19.646385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:19.859413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5b270b2ac14903a55936e1d5069d65'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:19.876213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5b270b2ac14903a55936e1d5069d65'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:19.949407 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5b270b2ac14903a55936e1d5069d65'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:19.950080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5b270b2ac14903a55936e1d5069d65'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:19.950698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5b270b2ac14903a55936e1d5069d65'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:19.963650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1223/2429] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:19 2022] DELETE /identity/v3/users/1e5b270b2ac14903a55936e1d5069d65 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:19.968902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb0c42-7503-4f8f-adfb-3fc15be56616 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:19.969371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb0c42-7503-4f8f-adfb-3fc15be56616 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:19.969371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb0c42-7503-4f8f-adfb-3fc15be56616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:19.990051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-39cb0c42-7503-4f8f-adfb-3fc15be56616 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:19.993930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-39cb0c42-7503-4f8f-adfb-3fc15be56616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:20.049991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:20.059718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1207/2430] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:20.074218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:20.074218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:20.133817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7a1bf-e0ba-46ae-a88d-e000bb639307 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:20.138278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a87f-251b-48ff-b688-b009e15c911a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:20.138887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a87f-251b-48ff-b688-b009e15c911a service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:20.139199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a87f-251b-48ff-b688-b009e15c911a service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:20.139494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a87f-251b-48ff-b688-b009e15c911a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:20.149215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a87f-251b-48ff-b688-b009e15c911a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:20.150770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a87f-251b-48ff-b688-b009e15c911a service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:20.162935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1224/2431] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:20.657920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb0c42-7503-4f8f-adfb-3fc15be56616 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:20.659820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-39cb0c42-7503-4f8f-adfb-3fc15be56616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:20.672991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb0c42-7503-4f8f-adfb-3fc15be56616 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:20.677659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:20.678331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:20.678642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:20.678922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] PATH_INFO: `/v3/projects/58cd72b7c46f42fe8b69aed3982fa8c8` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:20.680043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58cd72b7c46f42fe8b69aed3982fa8c8)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:20.682404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:20.839587 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58cd72b7c46f42fe8b69aed3982fa8c8', 'request_id': 'req-b7898d24-7432-4369-b690-ae628f7dc37d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:20.971207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58cd72b7c46f42fe8b69aed3982fa8c8', 'request_id': 'req-b7898d24-7432-4369-b690-ae628f7dc37d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:20.973149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58cd72b7c46f42fe8b69aed3982fa8c8', 'request_id': 'req-b7898d24-7432-4369-b690-ae628f7dc37d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:20.984047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] The token cache is being invalidate because project 58cd72b7c46f42fe8b69aed3982fa8c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:20.984431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:20.985115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:20.986704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1208/2432] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:20 2022] DELETE /identity/v3/projects/58cd72b7c46f42fe8b69aed3982fa8c8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:20.992349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a87f-251b-48ff-b688-b009e15c911a service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:21.014583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a87f-251b-48ff-b688-b009e15c911a service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:21.021441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:21.023066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:21.081154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:21.086572 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898d24-7432-4369-b690-ae628f7dc37d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:21.091702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:21.092371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:21.092653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:21.092930 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:21.095076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:21.099341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:21.100678 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:21.102623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:21.102623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] PATH_INFO: `/v3/users/e8f7a14f4ca04d6f96d0e54111c773f3` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:21.103031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f7a14f4ca04d6f96d0e54111c773f3)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:21.105511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:21.111226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:21.112729 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:21.130867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1209/2433] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:21.207253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:21.208688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:21.221278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8847b7-fef9-4b3b-9fc6-bb5c79452ef0 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:21.241882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:21.242536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:21.242828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:21.243085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:21.245026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:21.266022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:21.268252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:21.280673 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1210/2434] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:21.352266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f7a14f4ca04d6f96d0e54111c773f3'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:21.366275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f7a14f4ca04d6f96d0e54111c773f3'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:21.464961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f7a14f4ca04d6f96d0e54111c773f3'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:21.465611 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f7a14f4ca04d6f96d0e54111c773f3'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:21.466246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-2c6f7fbc-a222-480b-bdf8-586f14f24f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f7a14f4ca04d6f96d0e54111c773f3'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:21.484185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1225/2435] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:20 2022] DELETE /identity/v3/users/e8f7a14f4ca04d6f96d0e54111c773f3 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:22.696015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:22.698279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:22.720236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbd00d-428b-4be0-9f6a-20165ef920c0 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:22.799992 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:22.800760 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:22.801047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:22.801325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] PATH_INFO: `/v3/projects/42723667b25c4a1aa60ae1c6a0aeba11` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:22.802524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42723667b25c4a1aa60ae1c6a0aeba11)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:22.804607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:22.825059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:22.980849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42723667b25c4a1aa60ae1c6a0aeba11', 'request_id': 'req-46c77db1-9d40-4638-b81f-6040d0cb9840'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:23.050381 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42723667b25c4a1aa60ae1c6a0aeba11', 'request_id': 'req-46c77db1-9d40-4638-b81f-6040d0cb9840'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:23.060103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42723667b25c4a1aa60ae1c6a0aeba11', 'request_id': 'req-46c77db1-9d40-4638-b81f-6040d0cb9840'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:23.078594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] The token cache is being invalidate because project 42723667b25c4a1aa60ae1c6a0aeba11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:23.078923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:23.079627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-46c77db1-9d40-4638-b81f-6040d0cb9840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:23.081170 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1211/2436] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:22 2022] DELETE /identity/v3/projects/42723667b25c4a1aa60ae1c6a0aeba11 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:23.086721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7e16-4456-4682-9e13-74a8a5529a7a None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.088062 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7e16-4456-4682-9e13-74a8a5529a7a None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.088517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7e16-4456-4682-9e13-74a8a5529a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.121306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-40bf7e16-4456-4682-9e13-74a8a5529a7a None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:23.124412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-40bf7e16-4456-4682-9e13-74a8a5529a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:23.197544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1226/2437] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:23.265838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc4c21-647a-4ebb-9d13-2d31b954a3be None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.266168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc4c21-647a-4ebb-9d13-2d31b954a3be None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.267128 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc4c21-647a-4ebb-9d13-2d31b954a3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.287817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-38bc4c21-647a-4ebb-9d13-2d31b954a3be None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:23.289058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-38bc4c21-647a-4ebb-9d13-2d31b954a3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:23.303027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf7e16-4456-4682-9e13-74a8a5529a7a None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:23.303556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1212/2438] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:23.313363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf7e16-4456-4682-9e13-74a8a5529a7a None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:23.319841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:23.320464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.320751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.321045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.321714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:23.322058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:23.324303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:23.337203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1227/2439] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:23.342500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc4c21-647a-4ebb-9d13-2d31b954a3be None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:23.352860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc4c21-647a-4ebb-9d13-2d31b954a3be None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:23.358784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c6d9-ac83-46b5-88f3-11a2041a25e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:23.359280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c6d9-ac83-46b5-88f3-11a2041a25e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.359552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c6d9-ac83-46b5-88f3-11a2041a25e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.360260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c6d9-ac83-46b5-88f3-11a2041a25e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.360444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c6d9-ac83-46b5-88f3-11a2041a25e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:23.362435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c6d9-ac83-46b5-88f3-11a2041a25e9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:23.462766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1213/2440] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:23 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:23.464979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:23.477005 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e37c5c-5135-48de-b849-d61f404607cd admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:23.548169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:23.561574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26484a-0d52-45af-b6bd-62ba7a0be8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:23.561875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26484a-0d52-45af-b6bd-62ba7a0be8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.562049 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26484a-0d52-45af-b6bd-62ba7a0be8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.562324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26484a-0d52-45af-b6bd-62ba7a0be8f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.563548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26484a-0d52-45af-b6bd-62ba7a0be8f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:23.566096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26484a-0d52-45af-b6bd-62ba7a0be8f8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:23.730934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1228/2441] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:23 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:23.731086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c6d9-ac83-46b5-88f3-11a2041a25e9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:23.750868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c6d9-ac83-46b5-88f3-11a2041a25e9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:23.755751 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bacc5-22d0-456b-80ad-29434c6fba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:23.756393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bacc5-22d0-456b-80ad-29434c6fba51 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.759452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bacc5-22d0-456b-80ad-29434c6fba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.759452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bacc5-22d0-456b-80ad-29434c6fba51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.759452 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bacc5-22d0-456b-80ad-29434c6fba51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:23.760106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bacc5-22d0-456b-80ad-29434c6fba51 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:23.772448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1214/2442] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:23.778443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26484a-0d52-45af-b6bd-62ba7a0be8f8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:23.789311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26484a-0d52-45af-b6bd-62ba7a0be8f8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:23.795485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde23d-7d9b-4c69-b3fe-a57cebaea98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:23.796350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde23d-7d9b-4c69-b3fe-a57cebaea98f admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.796628 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde23d-7d9b-4c69-b3fe-a57cebaea98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.796963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde23d-7d9b-4c69-b3fe-a57cebaea98f admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.797708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde23d-7d9b-4c69-b3fe-a57cebaea98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:23.799520 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde23d-7d9b-4c69-b3fe-a57cebaea98f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:23.828070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1229/2443] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:23.832062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bacc5-22d0-456b-80ad-29434c6fba51 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:23.848547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bacc5-22d0-456b-80ad-29434c6fba51 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:23.856076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024823a1-697b-4d2a-aaae-c2bb10810bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:23.857249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024823a1-697b-4d2a-aaae-c2bb10810bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.857687 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024823a1-697b-4d2a-aaae-c2bb10810bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.858187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024823a1-697b-4d2a-aaae-c2bb10810bf9 admin admin] PATH_INFO: `/v3/projects/a29f4d36eaf74e06a506b650dea892d0/users/4ac4af8a35654cd4ac80199aed4800bc/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.889156 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024823a1-697b-4d2a-aaae-c2bb10810bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29f4d36eaf74e06a506b650dea892d0, user_id=4ac4af8a35654cd4ac80199aed4800bc, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:23.893472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024823a1-697b-4d2a-aaae-c2bb10810bf9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:23.976192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1215/2444] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:23 2022] PUT /identity/v3/projects/a29f4d36eaf74e06a506b650dea892d0/users/4ac4af8a35654cd4ac80199aed4800bc/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:23.986633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691868e-038f-4e46-896c-964f50933f47 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.986924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691868e-038f-4e46-896c-964f50933f47 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.987209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691868e-038f-4e46-896c-964f50933f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:24.021360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a691868e-038f-4e46-896c-964f50933f47 None None] MFA Rules not processed for user `4ac4af8a35654cd4ac80199aed4800bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:24.038870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a691868e-038f-4e46-896c-964f50933f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:24.204820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1230/2445] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:24.209347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024823a1-697b-4d2a-aaae-c2bb10810bf9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:24.349936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024823a1-697b-4d2a-aaae-c2bb10810bf9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:24.367977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86733d50-9aba-4bcb-83d2-ac747eb48b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:24.368611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86733d50-9aba-4bcb-83d2-ac747eb48b10 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:24.368905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86733d50-9aba-4bcb-83d2-ac747eb48b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:24.369162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86733d50-9aba-4bcb-83d2-ac747eb48b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:24.370398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86733d50-9aba-4bcb-83d2-ac747eb48b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:24.377099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86733d50-9aba-4bcb-83d2-ac747eb48b10 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:24.538352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1216/2446] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:24 2022] POST /identity/v3/projects => generated 381 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:24.539644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691868e-038f-4e46-896c-964f50933f47 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:24.556524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691868e-038f-4e46-896c-964f50933f47 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:24.653794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a791d-333b-4e8b-b32c-1db3c31bf888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:24.654522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a791d-333b-4e8b-b32c-1db3c31bf888 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:24.655060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a791d-333b-4e8b-b32c-1db3c31bf888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:24.655345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a791d-333b-4e8b-b32c-1db3c31bf888 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:24.656006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a791d-333b-4e8b-b32c-1db3c31bf888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:24.658004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a791d-333b-4e8b-b32c-1db3c31bf888 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:24.790354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1231/2447] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:24 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:24.795702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86733d50-9aba-4bcb-83d2-ac747eb48b10 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:24.806537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86733d50-9aba-4bcb-83d2-ac747eb48b10 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:24.812118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa97459-67f5-489e-88c6-09ee7f49b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:24.812726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa97459-67f5-489e-88c6-09ee7f49b97a admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:24.813008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa97459-67f5-489e-88c6-09ee7f49b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:24.813337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa97459-67f5-489e-88c6-09ee7f49b97a admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:24.814161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa97459-67f5-489e-88c6-09ee7f49b97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:24.816051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa97459-67f5-489e-88c6-09ee7f49b97a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:24.830027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1217/2448] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:24.832510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a791d-333b-4e8b-b32c-1db3c31bf888 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:24.843418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a791d-333b-4e8b-b32c-1db3c31bf888 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:24.850289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6749a89-4f33-4359-b11a-c40372015d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:24.850792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6749a89-4f33-4359-b11a-c40372015d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:24.851071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6749a89-4f33-4359-b11a-c40372015d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:24.851342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6749a89-4f33-4359-b11a-c40372015d0a admin admin] PATH_INFO: `/v3/projects/0b146c72181844368d6ff0d5c18a7128/users/3f88cc6664a747dda46e1d10b00ebcdb/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:24.869997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6749a89-4f33-4359-b11a-c40372015d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b146c72181844368d6ff0d5c18a7128, user_id=3f88cc6664a747dda46e1d10b00ebcdb, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:24.876105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6749a89-4f33-4359-b11a-c40372015d0a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:25.032487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa97459-67f5-489e-88c6-09ee7f49b97a admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:25.041203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1232/2449] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:24 2022] PUT /identity/v3/projects/0b146c72181844368d6ff0d5c18a7128/users/3f88cc6664a747dda46e1d10b00ebcdb/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:25.053695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa97459-67f5-489e-88c6-09ee7f49b97a admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:25.071983 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:25.179294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a11ca-52f3-4dc5-bf4e-d74335bf006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:25.180776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a11ca-52f3-4dc5-bf4e-d74335bf006e admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:25.180776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a11ca-52f3-4dc5-bf4e-d74335bf006e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:25.181088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a11ca-52f3-4dc5-bf4e-d74335bf006e admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:25.181551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a11ca-52f3-4dc5-bf4e-d74335bf006e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:25.184300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a11ca-52f3-4dc5-bf4e-d74335bf006e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:25.198964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1218/2450] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:25.204280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6749a89-4f33-4359-b11a-c40372015d0a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:25.226952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6749a89-4f33-4359-b11a-c40372015d0a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:25.232737 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:25.233550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:25.235125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:25.235125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] PATH_INFO: `/v3/projects/0b146c72181844368d6ff0d5c18a7128/users/3f88cc6664a747dda46e1d10b00ebcdb/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:25.236553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b146c72181844368d6ff0d5c18a7128, user_id=3f88cc6664a747dda46e1d10b00ebcdb, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:25.239484 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:25.253822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1233/2451] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:25 2022] PUT /identity/v3/projects/0b146c72181844368d6ff0d5c18a7128/users/3f88cc6664a747dda46e1d10b00ebcdb/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:25.265158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e1a29-2259-48cc-8fb2-e516dccf1b54 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:25.265429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e1a29-2259-48cc-8fb2-e516dccf1b54 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:25.265709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e1a29-2259-48cc-8fb2-e516dccf1b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:25.322825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2c0e1a29-2259-48cc-8fb2-e516dccf1b54 None None] MFA Rules not processed for user `3f88cc6664a747dda46e1d10b00ebcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:25.328519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2c0e1a29-2259-48cc-8fb2-e516dccf1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:25.462310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1219/2452] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:25.470403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:25.472261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:25.507471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:25.579344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42133583-0956-4464-8fab-9482ab16c386 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:25.584295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f846-9851-4231-b5d3-31090d9372aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:25.585326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f846-9851-4231-b5d3-31090d9372aa service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:25.586140 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f846-9851-4231-b5d3-31090d9372aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:25.588638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f846-9851-4231-b5d3-31090d9372aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:25.610227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2f846-9851-4231-b5d3-31090d9372aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:25.610667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2f846-9851-4231-b5d3-31090d9372aa service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:25.627819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1234/2453] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:25.742525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e1a29-2259-48cc-8fb2-e516dccf1b54 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:25.744710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2c0e1a29-2259-48cc-8fb2-e516dccf1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:25.808440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f846-9851-4231-b5d3-31090d9372aa service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:25.823997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f846-9851-4231-b5d3-31090d9372aa service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:25.828119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973908a2-f062-40cb-a744-2a1be43804ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:25.829547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973908a2-f062-40cb-a744-2a1be43804ec service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:25.830311 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973908a2-f062-40cb-a744-2a1be43804ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:25.830891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973908a2-f062-40cb-a744-2a1be43804ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:25.852338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e1a29-2259-48cc-8fb2-e516dccf1b54 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:25.859720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:25.864834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:25.864834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:25.864834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:25.864834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:25.879875 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:25.881722 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:25.897457 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1220/2454] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:25.934539 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973908a2-f062-40cb-a744-2a1be43804ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:25.936101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973908a2-f062-40cb-a744-2a1be43804ec service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:25.958457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1235/2455] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:25.981378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:25.983252 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:26.043056 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:26.105058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56541f93-59d2-4248-927a-b6389ca378ff service placement] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:26.113472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:26.114176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:26.114462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:26.114955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] PATH_INFO: `/v3/users/251bc85f89774142a97acd77895b14e9` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:26.116328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=251bc85f89774142a97acd77895b14e9)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:26.118512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:26.336858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '251bc85f89774142a97acd77895b14e9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:26.437824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '251bc85f89774142a97acd77895b14e9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:26.438859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '251bc85f89774142a97acd77895b14e9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:26.449385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '251bc85f89774142a97acd77895b14e9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:26.450140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-faa490e9-c9bc-4c83-b56b-80ba7f7c3f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '251bc85f89774142a97acd77895b14e9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:26.497661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1221/2456] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:25 2022] DELETE /identity/v3/users/251bc85f89774142a97acd77895b14e9 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:26.639785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973908a2-f062-40cb-a744-2a1be43804ec service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:26.652901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973908a2-f062-40cb-a744-2a1be43804ec service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:26.735277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:26.736149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:26.749150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:26.749150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] PATH_INFO: `/v3/users/4ac4af8a35654cd4ac80199aed4800bc` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:26.749150 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac4af8a35654cd4ac80199aed4800bc)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:26.750171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:26.935672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac4af8a35654cd4ac80199aed4800bc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:26.944979 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:26.951143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac4af8a35654cd4ac80199aed4800bc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.054648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac4af8a35654cd4ac80199aed4800bc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.055297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac4af8a35654cd4ac80199aed4800bc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.056330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac4af8a35654cd4ac80199aed4800bc'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.071708 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1236/2457] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:26 2022] DELETE /identity/v3/users/4ac4af8a35654cd4ac80199aed4800bc => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:27.077174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275995fe-d4f4-4f3e-92af-580dd4ff28b8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:27.078000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275995fe-d4f4-4f3e-92af-580dd4ff28b8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:27.078000 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275995fe-d4f4-4f3e-92af-580dd4ff28b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:27.099448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-275995fe-d4f4-4f3e-92af-580dd4ff28b8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:27.100784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-275995fe-d4f4-4f3e-92af-580dd4ff28b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:27.169207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:27.170799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:27.185840 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1222/2458] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:27.194819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275995fe-d4f4-4f3e-92af-580dd4ff28b8 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:27.213632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-275995fe-d4f4-4f3e-92af-580dd4ff28b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:27.228501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275995fe-d4f4-4f3e-92af-580dd4ff28b8 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:27.238631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:27.238631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:27.238631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:27.238631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] PATH_INFO: `/v3/projects/0a4d7fadea104c569ca29ff9be48089a` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:27.240389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4d7fadea104c569ca29ff9be48089a)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:27.242476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:27.290031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb7066-bb93-41a4-9cfe-b26ee5f6a683 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:27.295432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:27.296200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:27.296572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:27.296924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] PATH_INFO: `/v3/users/2b6b0d0cbdb54a46a7e9d6936c47cc4c` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:27.298118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6b0d0cbdb54a46a7e9d6936c47cc4c)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:27.300153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:27.391804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:27.395994 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4d7fadea104c569ca29ff9be48089a', 'request_id': 'req-e3930c3c-7420-408e-bad6-2a26aa47a385'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.497912 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4d7fadea104c569ca29ff9be48089a', 'request_id': 'req-e3930c3c-7420-408e-bad6-2a26aa47a385'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.506388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4d7fadea104c569ca29ff9be48089a', 'request_id': 'req-e3930c3c-7420-408e-bad6-2a26aa47a385'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.526089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] The token cache is being invalidate because project 0a4d7fadea104c569ca29ff9be48089a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:27.526355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.527231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.529027 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1223/2459] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:27 2022] DELETE /identity/v3/projects/0a4d7fadea104c569ca29ff9be48089a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:27.529684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:27.531091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:27.563332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6b0d0cbdb54a46a7e9d6936c47cc4c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.580329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6b0d0cbdb54a46a7e9d6936c47cc4c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.610753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930c3c-7420-408e-bad6-2a26aa47a385 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:27.616110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b78ae1-db09-4ed6-84a1-d1525f2e2154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:27.620528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b78ae1-db09-4ed6-84a1-d1525f2e2154 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:27.620528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b78ae1-db09-4ed6-84a1-d1525f2e2154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:27.620528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b78ae1-db09-4ed6-84a1-d1525f2e2154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:27.622718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b1b78ae1-db09-4ed6-84a1-d1525f2e2154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:27.698734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6b0d0cbdb54a46a7e9d6936c47cc4c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.699529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6b0d0cbdb54a46a7e9d6936c47cc4c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.700251 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-ffebf433-6658-4017-8d0b-89dd6cce72ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6b0d0cbdb54a46a7e9d6936c47cc4c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:27.707935 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b78ae1-db09-4ed6-84a1-d1525f2e2154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:27.714708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b78ae1-db09-4ed6-84a1-d1525f2e2154 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:27.723058 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1237/2460] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:27 2022] DELETE /identity/v3/users/2b6b0d0cbdb54a46a7e9d6936c47cc4c => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:27.724811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50e70e-338a-48e3-9223-49c916bbd899 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:27.735339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50e70e-338a-48e3-9223-49c916bbd899 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:27.735796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50e70e-338a-48e3-9223-49c916bbd899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:27.744437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1224/2461] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:27.769408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-6f50e70e-338a-48e3-9223-49c916bbd899 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:27.771604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6f50e70e-338a-48e3-9223-49c916bbd899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:27.856466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:27.891269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1238/2462] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:27.900320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94caff2-582c-4365-b79c-da410adeae9f None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:27.900617 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94caff2-582c-4365-b79c-da410adeae9f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:27.901009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94caff2-582c-4365-b79c-da410adeae9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:27.959477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e94caff2-582c-4365-b79c-da410adeae9f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:27.965702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e94caff2-582c-4365-b79c-da410adeae9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:27.977884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1225/2463] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:27.983502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50e70e-338a-48e3-9223-49c916bbd899 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:27.996598 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50e70e-338a-48e3-9223-49c916bbd899 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.001955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.002901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.003393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.003821 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.004641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:28.005149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.006943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.030490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1239/2464] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:56:28.033844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94caff2-582c-4365-b79c-da410adeae9f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.062932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94caff2-582c-4365-b79c-da410adeae9f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.086578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2fa50-cfaa-4530-81df-96062df10ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.087836 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2fa50-cfaa-4530-81df-96062df10ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.088117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2fa50-cfaa-4530-81df-96062df10ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.088694 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2fa50-cfaa-4530-81df-96062df10ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.089343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2fa50-cfaa-4530-81df-96062df10ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.091473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2fa50-cfaa-4530-81df-96062df10ad7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.197208 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.200486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:28.205445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1226/2465] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:28 2022] POST /identity/v3/projects => generated 407 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:28.209870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2fa50-cfaa-4530-81df-96062df10ad7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.226365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debeee6-a10e-43fa-846d-b57081d215e8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.296936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2fa50-cfaa-4530-81df-96062df10ad7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.301810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.302551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.302893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.303257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.303408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.304025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.304366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.304787 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.305202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] PATH_INFO: `/v3/projects/40256802535d4aedb2541246a687ec76` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.306264 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.306683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40256802535d4aedb2541246a687ec76)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.309663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.415564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1227/2466] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:28 2022] POST /identity/v3/users => generated 304 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:28.421175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.424317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:28.443508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40256802535d4aedb2541246a687ec76', 'request_id': 'req-bc6e5463-db21-4355-a895-3d0a982d4692'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.462853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb418e4c-d8f5-4045-bc21-4d350c9b7d7b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.540305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40256802535d4aedb2541246a687ec76', 'request_id': 'req-bc6e5463-db21-4355-a895-3d0a982d4692'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.543279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.544077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.544542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.544924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] PATH_INFO: `/v3/projects/a29f4d36eaf74e06a506b650dea892d0` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.546298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29f4d36eaf74e06a506b650dea892d0)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.548385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40256802535d4aedb2541246a687ec76', 'request_id': 'req-bc6e5463-db21-4355-a895-3d0a982d4692'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.548576 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.563724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] The token cache is being invalidate because project 40256802535d4aedb2541246a687ec76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:28.564082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.565604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.568282 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1240/2467] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:28 2022] DELETE /identity/v3/projects/40256802535d4aedb2541246a687ec76 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:28.569226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.571222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:28.586233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e5463-db21-4355-a895-3d0a982d4692 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.594769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ac68e-9cce-47ec-bb73-fd8601828224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.596232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ac68e-9cce-47ec-bb73-fd8601828224 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.596232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ac68e-9cce-47ec-bb73-fd8601828224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.596232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ac68e-9cce-47ec-bb73-fd8601828224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.597391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ac68e-9cce-47ec-bb73-fd8601828224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.599249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ac68e-9cce-47ec-bb73-fd8601828224 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.613380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1241/2468] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:28.614994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c034b-d050-49f3-bd0d-cf04b58e404b None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.615276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c034b-d050-49f3-bd0d-cf04b58e404b None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.615546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c034b-d050-49f3-bd0d-cf04b58e404b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.637811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-941c034b-d050-49f3-bd0d-cf04b58e404b None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:28.639163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-941c034b-d050-49f3-bd0d-cf04b58e404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:28.649891 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1242/2469] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:28.650333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c034b-d050-49f3-bd0d-cf04b58e404b None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.661732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c034b-d050-49f3-bd0d-cf04b58e404b None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.668119 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d9407-1757-44e1-9dbf-062eceb9f036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.668810 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d9407-1757-44e1-9dbf-062eceb9f036 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.669097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d9407-1757-44e1-9dbf-062eceb9f036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.669370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d9407-1757-44e1-9dbf-062eceb9f036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.670180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d9407-1757-44e1-9dbf-062eceb9f036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.671906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d9407-1757-44e1-9dbf-062eceb9f036 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.683896 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1243/2470] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:28.685032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cd33-9a07-4913-ac8d-016a21e8f68c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.685321 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cd33-9a07-4913-ac8d-016a21e8f68c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.685601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cd33-9a07-4913-ac8d-016a21e8f68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.705539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:28.709288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b989cd33-9a07-4913-ac8d-016a21e8f68c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:28.710635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b989cd33-9a07-4913-ac8d-016a21e8f68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:28.712603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29f4d36eaf74e06a506b650dea892d0', 'request_id': 'req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.775585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1244/2471] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:28.777394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989cd33-9a07-4913-ac8d-016a21e8f68c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.787108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989cd33-9a07-4913-ac8d-016a21e8f68c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.792697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55eb38-186d-4be0-bea0-5b6ffca4e6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.793352 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55eb38-186d-4be0-bea0-5b6ffca4e6ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.793633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55eb38-186d-4be0-bea0-5b6ffca4e6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.793992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55eb38-186d-4be0-bea0-5b6ffca4e6ab admin admin] PATH_INFO: `/v3/projects/10b541da08fa458b811cb0e8679d71ab/users/d37485f5cc7f4dd7bfdb0e4e6a846a13/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.798471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:28.808922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29f4d36eaf74e06a506b650dea892d0', 'request_id': 'req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.811862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55eb38-186d-4be0-bea0-5b6ffca4e6ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b541da08fa458b811cb0e8679d71ab, user_id=d37485f5cc7f4dd7bfdb0e4e6a846a13, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.814785 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55eb38-186d-4be0-bea0-5b6ffca4e6ab admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.816113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29f4d36eaf74e06a506b650dea892d0', 'request_id': 'req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.829614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] The token cache is being invalidate because project a29f4d36eaf74e06a506b650dea892d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:28.830023 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.830810 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:28.833436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1228/2472] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:28 2022] DELETE /identity/v3/projects/a29f4d36eaf74e06a506b650dea892d0 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:28.836676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.836676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:28.849221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b58aa4-9b2e-4cfd-afcb-9132c2b29702 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.854835 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.855440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.855723 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.856001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.856661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:28.856978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.858623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.872817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1229/2473] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:28.874055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.875860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:28.889562 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6ede8-35f3-4e13-9e62-67fb6e064ef5 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.894755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.895382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.895669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.895948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] PATH_INFO: `/v3/users/3f88cc6664a747dda46e1d10b00ebcdb` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.897130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f88cc6664a747dda46e1d10b00ebcdb)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.899076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:28.901068 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1245/2474] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:28 2022] PUT /identity/v3/projects/10b541da08fa458b811cb0e8679d71ab/users/d37485f5cc7f4dd7bfdb0e4e6a846a13/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:28.909166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55eb38-186d-4be0-bea0-5b6ffca4e6ab admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:28.978841 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55eb38-186d-4be0-bea0-5b6ffca4e6ab admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:28.987117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47715867-56da-43fa-8d74-c9d1145d6283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:28.987117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47715867-56da-43fa-8d74-c9d1145d6283 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:28.988817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47715867-56da-43fa-8d74-c9d1145d6283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:28.988817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47715867-56da-43fa-8d74-c9d1145d6283 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:28.988817 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47715867-56da-43fa-8d74-c9d1145d6283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:28.990763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47715867-56da-43fa-8d74-c9d1145d6283 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.101123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1246/2475] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:28 2022] POST /identity/v3/projects => generated 363 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:29.101123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1535da-7cc8-4f41-958f-d5be43c1b778 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.101123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1535da-7cc8-4f41-958f-d5be43c1b778 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.101123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1535da-7cc8-4f41-958f-d5be43c1b778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.133748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-5c1535da-7cc8-4f41-958f-d5be43c1b778 None None] MFA Rules not processed for user `d37485f5cc7f4dd7bfdb0e4e6a846a13`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:29.135013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5c1535da-7cc8-4f41-958f-d5be43c1b778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:29.162102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f88cc6664a747dda46e1d10b00ebcdb'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:29.216869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1247/2476] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:29 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:29.217673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1535da-7cc8-4f41-958f-d5be43c1b778 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.278813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f88cc6664a747dda46e1d10b00ebcdb'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:29.283634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:29.283634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f88cc6664a747dda46e1d10b00ebcdb'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:29.285201 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1535da-7cc8-4f41-958f-d5be43c1b778 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.291530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f88cc6664a747dda46e1d10b00ebcdb'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:29.291750 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.292161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.292517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.292837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.293039 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f88cc6664a747dda46e1d10b00ebcdb'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:29.294192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.296241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.314418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1230/2477] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:28 2022] DELETE /identity/v3/users/3f88cc6664a747dda46e1d10b00ebcdb => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:29.315327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.316802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:29.384561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65a23-6366-4e1b-9290-1df341610378 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.389946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64128c40-b085-4200-8de5-e15174fd657f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.390592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64128c40-b085-4200-8de5-e15174fd657f admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.390880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64128c40-b085-4200-8de5-e15174fd657f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.391284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64128c40-b085-4200-8de5-e15174fd657f admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.392182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64128c40-b085-4200-8de5-e15174fd657f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.393424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:29.394187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64128c40-b085-4200-8de5-e15174fd657f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.439884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1248/2478] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:29 2022] POST /identity/v3/users => generated 282 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:29.444384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.446195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:29.459986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189325f-8af9-4bf5-a8a2-32c7a05013f0 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.466277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb7b75-eb8f-49bc-a59b-cdb2585ca9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.467012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb7b75-eb8f-49bc-a59b-cdb2585ca9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.470192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb7b75-eb8f-49bc-a59b-cdb2585ca9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.470192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb7b75-eb8f-49bc-a59b-cdb2585ca9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.470192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb7b75-eb8f-49bc-a59b-cdb2585ca9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.470192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb7b75-eb8f-49bc-a59b-cdb2585ca9b1 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.481379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1249/2479] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:29.485530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb7b75-eb8f-49bc-a59b-cdb2585ca9b1 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.496480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb7b75-eb8f-49bc-a59b-cdb2585ca9b1 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.501177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a019-964b-48f2-88ca-27d77c09e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.501800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a019-964b-48f2-88ca-27d77c09e623 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.504449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a019-964b-48f2-88ca-27d77c09e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.504449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a019-964b-48f2-88ca-27d77c09e623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.504449 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108a019-964b-48f2-88ca-27d77c09e623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.504881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108a019-964b-48f2-88ca-27d77c09e623 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.516380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1250/2480] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:29.521088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a019-964b-48f2-88ca-27d77c09e623 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.532384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a019-964b-48f2-88ca-27d77c09e623 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.537268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03855fac-3406-4423-9bad-a853247ad0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.544718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855fac-3406-4423-9bad-a853247ad0b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.544718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855fac-3406-4423-9bad-a853247ad0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.544718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855fac-3406-4423-9bad-a853247ad0b4 admin admin] PATH_INFO: `/v3/projects/118f06781ae541efadc42611b29639ab/users/78c70a5030bd495cb467e8919bb0eb19/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.554902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03855fac-3406-4423-9bad-a853247ad0b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118f06781ae541efadc42611b29639ab, user_id=78c70a5030bd495cb467e8919bb0eb19, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.558458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03855fac-3406-4423-9bad-a853247ad0b4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.588042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1231/2481] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:29 2022] POST /identity/v3/projects => generated 409 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:29.592253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64128c40-b085-4200-8de5-e15174fd657f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.660405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1251/2482] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:29 2022] PUT /identity/v3/projects/118f06781ae541efadc42611b29639ab/users/78c70a5030bd495cb467e8919bb0eb19/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:29.679029 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e616-b86b-4278-aaa6-782041838e35 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.679418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e616-b86b-4278-aaa6-782041838e35 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.679863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e616-b86b-4278-aaa6-782041838e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.681656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64128c40-b085-4200-8de5-e15174fd657f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.748137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c589e616-b86b-4278-aaa6-782041838e35 None None] MFA Rules not processed for user `78c70a5030bd495cb467e8919bb0eb19`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:29.749000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c589e616-b86b-4278-aaa6-782041838e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:29.765904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a3e3c-2830-4991-be57-686100820ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.766923 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a3e3c-2830-4991-be57-686100820ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.767203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a3e3c-2830-4991-be57-686100820ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.768054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a3e3c-2830-4991-be57-686100820ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.769385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a3e3c-2830-4991-be57-686100820ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.771585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a3e3c-2830-4991-be57-686100820ad8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.842278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1252/2483] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:29.848102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589e616-b86b-4278-aaa6-782041838e35 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.860603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589e616-b86b-4278-aaa6-782041838e35 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.865907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.866658 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.867051 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.867416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.868366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.871137 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.877900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1232/2484] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:29 2022] POST /identity/v3/users => generated 311 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:29.879789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a3e3c-2830-4991-be57-686100820ad8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.892052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a3e3c-2830-4991-be57-686100820ad8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.897165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172868f-8ef9-4a84-89ab-6468df082ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.897886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172868f-8ef9-4a84-89ab-6468df082ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.900124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172868f-8ef9-4a84-89ab-6468df082ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.900124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172868f-8ef9-4a84-89ab-6468df082ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.900124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172868f-8ef9-4a84-89ab-6468df082ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.900929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172868f-8ef9-4a84-89ab-6468df082ac1 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.919107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1233/2485] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:29.920552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172868f-8ef9-4a84-89ab-6468df082ac1 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.930560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172868f-8ef9-4a84-89ab-6468df082ac1 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:29.936605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af352df0-4743-4197-ad45-4b8a367eaa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:29.937253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af352df0-4743-4197-ad45-4b8a367eaa19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:29.937533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af352df0-4743-4197-ad45-4b8a367eaa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:29.937802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af352df0-4743-4197-ad45-4b8a367eaa19 admin admin] PATH_INFO: `/v3/projects/abccb3aed9cf4a3987b2ec78eef26b9e/users/49d590c2adfe43df82303f321c3cc307/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:29.972213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af352df0-4743-4197-ad45-4b8a367eaa19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abccb3aed9cf4a3987b2ec78eef26b9e, user_id=49d590c2adfe43df82303f321c3cc307, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:29.975119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af352df0-4743-4197-ad45-4b8a367eaa19 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:29.985024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:29.997793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1253/2486] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:29 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:29.997793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:29.997793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.058721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:30.077982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66de0b-ecb2-4713-8009-e8415b1a5263 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.082733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.083368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.083648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.083918 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] PATH_INFO: `/v3/projects/0b146c72181844368d6ff0d5c18a7128` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.085232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b146c72181844368d6ff0d5c18a7128)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.087185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.093269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1234/2487] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:29 2022] PUT /identity/v3/projects/abccb3aed9cf4a3987b2ec78eef26b9e/users/49d590c2adfe43df82303f321c3cc307/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:30.097339 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af352df0-4743-4197-ad45-4b8a367eaa19 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.161019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af352df0-4743-4197-ad45-4b8a367eaa19 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.169721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0009e2-84e6-40ca-9c80-894bca796b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.170413 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0009e2-84e6-40ca-9c80-894bca796b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.170641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0009e2-84e6-40ca-9c80-894bca796b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.170906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0009e2-84e6-40ca-9c80-894bca796b76 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.171534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0009e2-84e6-40ca-9c80-894bca796b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.173480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0009e2-84e6-40ca-9c80-894bca796b76 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.219398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b146c72181844368d6ff0d5c18a7128', 'request_id': 'req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.316985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1235/2488] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:30 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:30.316985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0009e2-84e6-40ca-9c80-894bca796b76 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.327545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0009e2-84e6-40ca-9c80-894bca796b76 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.347082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b146c72181844368d6ff0d5c18a7128', 'request_id': 'req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.355824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b146c72181844368d6ff0d5c18a7128', 'request_id': 'req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.366743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] The token cache is being invalidate because project 0b146c72181844368d6ff0d5c18a7128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:30.367033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.367715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.369583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1254/2489] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:29 2022] DELETE /identity/v3/projects/0b146c72181844368d6ff0d5c18a7128 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:30.370173 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.371496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.392343 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.393043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.393319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.393592 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.394732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.396357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.409538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1236/2490] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:30.414790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.417665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.433324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3e521-4075-44b9-94ea-0a21ad9ba6e3 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.439335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859e7dc-ad28-4e34-9fb7-ca86d2a30aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.440284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859e7dc-ad28-4e34-9fb7-ca86d2a30aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.441237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859e7dc-ad28-4e34-9fb7-ca86d2a30aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.441237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859e7dc-ad28-4e34-9fb7-ca86d2a30aa8 admin admin] PATH_INFO: `/v3/projects/abccb3aed9cf4a3987b2ec78eef26b9e/users/49d590c2adfe43df82303f321c3cc307/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.443269 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859e7dc-ad28-4e34-9fb7-ca86d2a30aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abccb3aed9cf4a3987b2ec78eef26b9e, user_id=49d590c2adfe43df82303f321c3cc307, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.446513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859e7dc-ad28-4e34-9fb7-ca86d2a30aa8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.455111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bdeca-b3fd-4dc2-9191-451acb7a4fdb admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.460574 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1237/2491] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:30 2022] PUT /identity/v3/projects/abccb3aed9cf4a3987b2ec78eef26b9e/users/49d590c2adfe43df82303f321c3cc307/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:30.473539 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a2cb-9dc2-4671-a8c9-9dbcaf33be78 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.473915 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a2cb-9dc2-4671-a8c9-9dbcaf33be78 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.474291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a2cb-9dc2-4671-a8c9-9dbcaf33be78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.526668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-cea9a2cb-9dc2-4671-a8c9-9dbcaf33be78 None None] MFA Rules not processed for user `49d590c2adfe43df82303f321c3cc307`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:30.529716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cea9a2cb-9dc2-4671-a8c9-9dbcaf33be78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.544773 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385707c-ed58-4a7d-bace-cf0e64fe9364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.545369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385707c-ed58-4a7d-bace-cf0e64fe9364 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.545702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385707c-ed58-4a7d-bace-cf0e64fe9364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.545981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385707c-ed58-4a7d-bace-cf0e64fe9364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.546692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385707c-ed58-4a7d-bace-cf0e64fe9364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.548742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385707c-ed58-4a7d-bace-cf0e64fe9364 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.564314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1255/2492] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:30.569056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385707c-ed58-4a7d-bace-cf0e64fe9364 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.585097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385707c-ed58-4a7d-bace-cf0e64fe9364 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.605504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189a2cb-f3a1-43b1-a272-f21ac9d21134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.606862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a2cb-f3a1-43b1-a272-f21ac9d21134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.607317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a2cb-f3a1-43b1-a272-f21ac9d21134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.607694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a2cb-f3a1-43b1-a272-f21ac9d21134 admin admin] PATH_INFO: `/v3/projects/da41b0c2b11142d3a154fc79cda259d5/users/49320a0b85b34223b058bc8d504dd7f5/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.631350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189a2cb-f3a1-43b1-a272-f21ac9d21134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da41b0c2b11142d3a154fc79cda259d5, user_id=49320a0b85b34223b058bc8d504dd7f5, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.638548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189a2cb-f3a1-43b1-a272-f21ac9d21134 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.643216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:30.646143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1238/2493] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:30 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:30.658054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a2cb-9dc2-4671-a8c9-9dbcaf33be78 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.659577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-cea9a2cb-9dc2-4671-a8c9-9dbcaf33be78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.680355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:30.706668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1256/2494] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:30 2022] PUT /identity/v3/projects/da41b0c2b11142d3a154fc79cda259d5/users/49320a0b85b34223b058bc8d504dd7f5/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:30.710711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189a2cb-f3a1-43b1-a272-f21ac9d21134 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.725758 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a2cb-9dc2-4671-a8c9-9dbcaf33be78 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.785113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189a2cb-f3a1-43b1-a272-f21ac9d21134 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.785757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0856f1-25cb-4b21-ae30-992f711d369b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.786351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0856f1-25cb-4b21-ae30-992f711d369b service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.786628 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0856f1-25cb-4b21-ae30-992f711d369b service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.786904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0856f1-25cb-4b21-ae30-992f711d369b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.790876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21953351-1df6-4af3-812f-af79c457d8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.792037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21953351-1df6-4af3-812f-af79c457d8ad admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.792462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21953351-1df6-4af3-812f-af79c457d8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.792745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21953351-1df6-4af3-812f-af79c457d8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.793516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21953351-1df6-4af3-812f-af79c457d8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.795250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21953351-1df6-4af3-812f-af79c457d8ad admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.806928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1257/2495] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:30.811540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21953351-1df6-4af3-812f-af79c457d8ad admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.821602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21953351-1df6-4af3-812f-af79c457d8ad admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.829219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cd2c2-f891-41bd-a52c-89b1e6535a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.831879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd2c2-f891-41bd-a52c-89b1e6535a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.831879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd2c2-f891-41bd-a52c-89b1e6535a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.831879 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd2c2-f891-41bd-a52c-89b1e6535a65 admin admin] PATH_INFO: `/v3/projects/da41b0c2b11142d3a154fc79cda259d5/users/49320a0b85b34223b058bc8d504dd7f5/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.833007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cd2c2-f891-41bd-a52c-89b1e6535a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da41b0c2b11142d3a154fc79cda259d5, user_id=49320a0b85b34223b058bc8d504dd7f5, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.835778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cd2c2-f891-41bd-a52c-89b1e6535a65 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.846555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1258/2496] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:30 2022] PUT /identity/v3/projects/da41b0c2b11142d3a154fc79cda259d5/users/49320a0b85b34223b058bc8d504dd7f5/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:30.850455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0856f1-25cb-4b21-ae30-992f711d369b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.851988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0856f1-25cb-4b21-ae30-992f711d369b service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.858355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d689a6-c432-44a8-88f3-c94eabe6d613 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.860644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d689a6-c432-44a8-88f3-c94eabe6d613 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.860644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d689a6-c432-44a8-88f3-c94eabe6d613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.863143 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1239/2497] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:30 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:30.890247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d3d689a6-c432-44a8-88f3-c94eabe6d613 None None] MFA Rules not processed for user `49320a0b85b34223b058bc8d504dd7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:30.892032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d3d689a6-c432-44a8-88f3-c94eabe6d613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.983966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1259/2498] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:31.001957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0856f1-25cb-4b21-ae30-992f711d369b service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:31.011861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0856f1-25cb-4b21-ae30-992f711d369b service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:31.067108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:31.067701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:31.067977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:31.068304 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:31.081205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:31.082705 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:31.093082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1240/2499] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:31.307660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d689a6-c432-44a8-88f3-c94eabe6d613 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:31.318393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d689a6-c432-44a8-88f3-c94eabe6d613 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:31.323715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:31.324534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:31.324880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:31.325349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:31.327264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:31.333641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:31.335246 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:31.347987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4313080-3320-4841-b733-c1953ff1ec04 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:31.352052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:31.352784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:31.353097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:31.353732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:31.356488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:31.378239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:31.396364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:31.397877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:31.407870 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1260/2500] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:31.425050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:31.425245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:31.427133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:31.439006 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1241/2501] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:31 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:33.483260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:33.484750 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:33.550511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab21b24-147c-43b3-b9d1-caceba130d05 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:33.555447 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9bd7-a009-461d-b498-c686bf4ddcc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:33.556071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9bd7-a009-461d-b498-c686bf4ddcc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:33.556348 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9bd7-a009-461d-b498-c686bf4ddcc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:33.556643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9bd7-a009-461d-b498-c686bf4ddcc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:33.559222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-086b9bd7-a009-461d-b498-c686bf4ddcc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:33.636569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9bd7-a009-461d-b498-c686bf4ddcc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:33.638048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9bd7-a009-461d-b498-c686bf4ddcc7 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:33.655196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1261/2502] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:35.119169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:35.132103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452362-16c2-4258-a011-5186711df55a service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:35.144919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:35.145606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:35.145916 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:35.146384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] PATH_INFO: `/v3/users/78c70a5030bd495cb467e8919bb0eb19` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:35.147581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c70a5030bd495cb467e8919bb0eb19)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:35.149668 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:35.329258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c70a5030bd495cb467e8919bb0eb19'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:35.389914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c70a5030bd495cb467e8919bb0eb19'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:35.390665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c70a5030bd495cb467e8919bb0eb19'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:35.402387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c70a5030bd495cb467e8919bb0eb19'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:35.409929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c70a5030bd495cb467e8919bb0eb19'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:35.427450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1242/2503] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:35 2022] DELETE /identity/v3/users/78c70a5030bd495cb467e8919bb0eb19 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:35.432560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0840-6ab6-498f-a61b-b1ce377249da None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:35.432560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0840-6ab6-498f-a61b-b1ce377249da None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:35.433779 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0840-6ab6-498f-a61b-b1ce377249da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:35.455105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-fd2b0840-6ab6-498f-a61b-b1ce377249da None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:35.456567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-fd2b0840-6ab6-498f-a61b-b1ce377249da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:35.530938 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1262/2504] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:35.546591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:35.548428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:35.588138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:35.613088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b262b1-024b-4168-b110-bf463495444b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:35.617263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025aa9-6472-4587-ae77-f35cc503eafa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:35.618063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025aa9-6472-4587-ae77-f35cc503eafa service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:35.618447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025aa9-6472-4587-ae77-f35cc503eafa service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:35.618822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025aa9-6472-4587-ae77-f35cc503eafa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:35.631191 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74025aa9-6472-4587-ae77-f35cc503eafa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:35.639742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74025aa9-6472-4587-ae77-f35cc503eafa service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:35.645407 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1243/2505] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:36.176633 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0840-6ab6-498f-a61b-b1ce377249da None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:36.178087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-fd2b0840-6ab6-498f-a61b-b1ce377249da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:36.192843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0840-6ab6-498f-a61b-b1ce377249da None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:36.199078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:36.199940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:36.200245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:36.200770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] PATH_INFO: `/v3/projects/118f06781ae541efadc42611b29639ab` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:36.207059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=118f06781ae541efadc42611b29639ab)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:36.209022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:36.358655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:36.361621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118f06781ae541efadc42611b29639ab', 'request_id': 'req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.467559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118f06781ae541efadc42611b29639ab', 'request_id': 'req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.477547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118f06781ae541efadc42611b29639ab', 'request_id': 'req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.490548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] The token cache is being invalidate because project 118f06781ae541efadc42611b29639ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:36.490834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.491514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.493222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1263/2506] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:36 2022] DELETE /identity/v3/projects/118f06781ae541efadc42611b29639ab => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:36.498057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025aa9-6472-4587-ae77-f35cc503eafa service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:36.510192 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025aa9-6472-4587-ae77-f35cc503eafa service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:36.578428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:36.579219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:36.579602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:36.580018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] PATH_INFO: `/v3/users/49320a0b85b34223b058bc8d504dd7f5` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:36.581422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49320a0b85b34223b058bc8d504dd7f5)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:36.583511 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:36.800045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49320a0b85b34223b058bc8d504dd7f5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.872768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49320a0b85b34223b058bc8d504dd7f5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.873941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49320a0b85b34223b058bc8d504dd7f5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.884939 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49320a0b85b34223b058bc8d504dd7f5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.885976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-68a06caa-0c90-4677-b6fd-b8a3bcafff30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49320a0b85b34223b058bc8d504dd7f5'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:36.924098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1244/2507] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:36 2022] DELETE /identity/v3/users/49320a0b85b34223b058bc8d504dd7f5 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:37.415702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:37.415702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:37.431312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5e99f-cf9e-4f83-a357-bfcca6b32b39 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:37.505331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:37.506932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:37.506932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:37.506932 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] PATH_INFO: `/v3/projects/da41b0c2b11142d3a154fc79cda259d5` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:37.508043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=da41b0c2b11142d3a154fc79cda259d5)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:37.510157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:37.697588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da41b0c2b11142d3a154fc79cda259d5', 'request_id': 'req-db08e719-9d28-496f-9952-c98ee9b727cd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:37.815595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da41b0c2b11142d3a154fc79cda259d5', 'request_id': 'req-db08e719-9d28-496f-9952-c98ee9b727cd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:37.826356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da41b0c2b11142d3a154fc79cda259d5', 'request_id': 'req-db08e719-9d28-496f-9952-c98ee9b727cd'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:37.839473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] The token cache is being invalidate because project da41b0c2b11142d3a154fc79cda259d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:37.839759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:37.841077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:37.846395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1264/2508] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:37 2022] DELETE /identity/v3/projects/da41b0c2b11142d3a154fc79cda259d5 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:37.848059 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8728eb2-8a7d-46d5-98cc-f6bca42d01e3 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:37.848683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8728eb2-8a7d-46d5-98cc-f6bca42d01e3 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:37.848683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8728eb2-8a7d-46d5-98cc-f6bca42d01e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:37.869734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a8728eb2-8a7d-46d5-98cc-f6bca42d01e3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:37.871183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a8728eb2-8a7d-46d5-98cc-f6bca42d01e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:37.890884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:37.892700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:37.939367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:37.954754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1245/2509] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:37.964461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f86fda-408d-4f53-9b77-fd95f59899ab None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:37.968396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f86fda-408d-4f53-9b77-fd95f59899ab None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:37.968396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f86fda-408d-4f53-9b77-fd95f59899ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:37.986976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-78f86fda-408d-4f53-9b77-fd95f59899ab None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:37.988206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-78f86fda-408d-4f53-9b77-fd95f59899ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:37.992963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08e719-9d28-496f-9952-c98ee9b727cd admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:37.998034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1246/2510] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:37.998546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:37.999289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:37.999647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:37.999978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] PATH_INFO: `/v3/users/612dfebb8ebc401bb81d3d3333577566` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.001363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612dfebb8ebc401bb81d3d3333577566)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.002914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f86fda-408d-4f53-9b77-fd95f59899ab None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:38.003478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.010538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:38.014319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f86fda-408d-4f53-9b77-fd95f59899ab None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:38.021329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:38.026787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.026787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.026787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.026787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:38.026787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.026787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.043514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1247/2511] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:38.047906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:38.058390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463d512-19ad-4940-b762-ae9763c132f6 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:38.063832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150e8fd-f2e0-440b-b273-a502444b4259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:38.064432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150e8fd-f2e0-440b-b273-a502444b4259 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.064847 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150e8fd-f2e0-440b-b273-a502444b4259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.065129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150e8fd-f2e0-440b-b273-a502444b4259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.065754 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150e8fd-f2e0-440b-b273-a502444b4259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.067747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150e8fd-f2e0-440b-b273-a502444b4259 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.179868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1248/2512] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:38 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:38.185055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150e8fd-f2e0-440b-b273-a502444b4259 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:38.217567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612dfebb8ebc401bb81d3d3333577566'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.233338 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612dfebb8ebc401bb81d3d3333577566'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.265230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150e8fd-f2e0-440b-b273-a502444b4259 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:38.268538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05c565-10e0-454e-a78c-e027c518ea23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:38.269172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05c565-10e0-454e-a78c-e027c518ea23 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.269462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05c565-10e0-454e-a78c-e027c518ea23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.269736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05c565-10e0-454e-a78c-e027c518ea23 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.270405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05c565-10e0-454e-a78c-e027c518ea23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.272354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05c565-10e0-454e-a78c-e027c518ea23 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.368706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612dfebb8ebc401bb81d3d3333577566'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.369339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612dfebb8ebc401bb81d3d3333577566'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.369980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612dfebb8ebc401bb81d3d3333577566'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.383572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1265/2513] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:37 2022] DELETE /identity/v3/users/612dfebb8ebc401bb81d3d3333577566 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:38.398985 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1249/2514] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:38 2022] POST /identity/v3/users => generated 285 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:38.402952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:38.404443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:38.476609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60427-076b-4d11-85bc-09350c701180 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:38.482088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc46f6-257b-4d9e-bb3a-69fd87a705b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:38.482620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc46f6-257b-4d9e-bb3a-69fd87a705b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.482902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc46f6-257b-4d9e-bb3a-69fd87a705b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.483183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc46f6-257b-4d9e-bb3a-69fd87a705b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.483848 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc46f6-257b-4d9e-bb3a-69fd87a705b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.485527 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc46f6-257b-4d9e-bb3a-69fd87a705b4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.496258 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1266/2515] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:38.501114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05c565-10e0-454e-a78c-e027c518ea23 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:38.511820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05c565-10e0-454e-a78c-e027c518ea23 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:38.517280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:38.532441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.532441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.532441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.532441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.532441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.532441 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1250/2516] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:38.536456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc46f6-257b-4d9e-bb3a-69fd87a705b4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:38.548939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc46f6-257b-4d9e-bb3a-69fd87a705b4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:38.555546 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccacf1-0be5-4918-b5de-8bbce4cd574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:38.556565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccacf1-0be5-4918-b5de-8bbce4cd574c admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.556947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccacf1-0be5-4918-b5de-8bbce4cd574c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.557440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccacf1-0be5-4918-b5de-8bbce4cd574c admin admin] PATH_INFO: `/v3/projects/c58b2fa0b4e4476ea77070b96e03a82e/users/e45bef4ec09c430395598bae9a0dd598/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.574939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccacf1-0be5-4918-b5de-8bbce4cd574c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58b2fa0b4e4476ea77070b96e03a82e, user_id=e45bef4ec09c430395598bae9a0dd598, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.577788 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccacf1-0be5-4918-b5de-8bbce4cd574c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.639904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:38.642079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:38.656332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96075c4-9e73-42f3-b536-f8de7c87fc1c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:38.661360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:38.662031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.662319 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.662613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] PATH_INFO: `/v3/projects/9398272c7a3f4a50a38df4b5dec9ac90` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.663790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9398272c7a3f4a50a38df4b5dec9ac90)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.665811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.695263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1267/2517] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:38 2022] PUT /identity/v3/projects/c58b2fa0b4e4476ea77070b96e03a82e/users/e45bef4ec09c430395598bae9a0dd598/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:38.706354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec738-0b77-41e9-b402-699a6b520036 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.706675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec738-0b77-41e9-b402-699a6b520036 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.706955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec738-0b77-41e9-b402-699a6b520036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.735506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-cbbec738-0b77-41e9-b402-699a6b520036 None None] MFA Rules not processed for user `e45bef4ec09c430395598bae9a0dd598`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:38.736751 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cbbec738-0b77-41e9-b402-699a6b520036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:38.783042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:38.794214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9398272c7a3f4a50a38df4b5dec9ac90', 'request_id': 'req-6a8e3ea5-33fb-412a-9785-8e9a026d595e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.806083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1268/2518] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:38.811091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbec738-0b77-41e9-b402-699a6b520036 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:38.848332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9398272c7a3f4a50a38df4b5dec9ac90', 'request_id': 'req-6a8e3ea5-33fb-412a-9785-8e9a026d595e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.854808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9398272c7a3f4a50a38df4b5dec9ac90', 'request_id': 'req-6a8e3ea5-33fb-412a-9785-8e9a026d595e'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.868556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] The token cache is being invalidate because project 9398272c7a3f4a50a38df4b5dec9ac90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:38.868849 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.869537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6a8e3ea5-33fb-412a-9785-8e9a026d595e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:38.871097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1251/2519] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:38 2022] DELETE /identity/v3/projects/9398272c7a3f4a50a38df4b5dec9ac90 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:38.876835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbec738-0b77-41e9-b402-699a6b520036 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:38.877024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7548-b1ac-498e-98f5-f7cc27faee22 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.877129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7548-b1ac-498e-98f5-f7cc27faee22 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.877405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7548-b1ac-498e-98f5-f7cc27faee22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.897563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-68bd7548-b1ac-498e-98f5-f7cc27faee22 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:38.899298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-68bd7548-b1ac-498e-98f5-f7cc27faee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:38.912169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:38.940198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:38.940812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.941092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.941367 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:38.942034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:38.944030 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:38.970963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1252/2520] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:38.979244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c976bf-9c84-4f11-85a9-03505acd4f8e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:38.984614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c976bf-9c84-4f11-85a9-03505acd4f8e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:38.984614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c976bf-9c84-4f11-85a9-03505acd4f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.001177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-41c976bf-9c84-4f11-85a9-03505acd4f8e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:39.002454 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-41c976bf-9c84-4f11-85a9-03505acd4f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:39.010641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1253/2521] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:39.015625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c976bf-9c84-4f11-85a9-03505acd4f8e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.024943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c976bf-9c84-4f11-85a9-03505acd4f8e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.029988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.030582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.030864 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.031139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.031821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:39.032126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.033818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.037500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1269/2522] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:38 2022] POST /identity/v3/projects => generated 369 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:39.042229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.044006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:39.049367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1254/2523] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:56:39.050365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.060719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e7f94-c6e4-48a9-b16e-e6bda2e81837 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.113822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a24c92-d50a-4d7d-bb18-e7b0886f8ba5 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.118437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469e954-f025-49fc-ace9-63d4fbb1dd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.118835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2f88-be31-4ca9-aee6-5da6c69dce18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.119047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469e954-f025-49fc-ace9-63d4fbb1dd55 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.119333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469e954-f025-49fc-ace9-63d4fbb1dd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.119433 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2f88-be31-4ca9-aee6-5da6c69dce18 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.119604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469e954-f025-49fc-ace9-63d4fbb1dd55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.119702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2f88-be31-4ca9-aee6-5da6c69dce18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.119978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2f88-be31-4ca9-aee6-5da6c69dce18 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.120229 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469e954-f025-49fc-ace9-63d4fbb1dd55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.120637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e2f88-be31-4ca9-aee6-5da6c69dce18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.122946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469e954-f025-49fc-ace9-63d4fbb1dd55 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.123175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e2f88-be31-4ca9-aee6-5da6c69dce18 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.226528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1270/2524] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:39 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:39.231757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2f88-be31-4ca9-aee6-5da6c69dce18 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.241833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2f88-be31-4ca9-aee6-5da6c69dce18 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.246496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5d99-3eea-40e3-be73-6e086fb96411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.247983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5d99-3eea-40e3-be73-6e086fb96411 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.248265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5d99-3eea-40e3-be73-6e086fb96411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.248553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5d99-3eea-40e3-be73-6e086fb96411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.249176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef5d99-3eea-40e3-be73-6e086fb96411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.250776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef5d99-3eea-40e3-be73-6e086fb96411 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.252121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:39.258451 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1255/2525] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:39 2022] POST /identity/v3/projects => generated 369 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:39.262136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469e954-f025-49fc-ace9-63d4fbb1dd55 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.265615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1271/2526] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:39.266814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5d99-3eea-40e3-be73-6e086fb96411 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.277147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5d99-3eea-40e3-be73-6e086fb96411 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.339372 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469e954-f025-49fc-ace9-63d4fbb1dd55 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.343707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b3ee8-64da-4e65-8a2f-74b5b5a63ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.344405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b3ee8-64da-4e65-8a2f-74b5b5a63ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.344526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b85e0-2b06-45a9-a062-47c250c397c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.344804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b3ee8-64da-4e65-8a2f-74b5b5a63ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.344919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b3ee8-64da-4e65-8a2f-74b5b5a63ed6 admin admin] PATH_INFO: `/v3/projects/dd7bda2c0e354724a43506ff00be8eac/users/a5de257ac9db41c28ac793891661c1a1/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.345015 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b85e0-2b06-45a9-a062-47c250c397c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.345280 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b85e0-2b06-45a9-a062-47c250c397c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.345594 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b85e0-2b06-45a9-a062-47c250c397c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.346400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b85e0-2b06-45a9-a062-47c250c397c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.348323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b85e0-2b06-45a9-a062-47c250c397c8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.361171 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b3ee8-64da-4e65-8a2f-74b5b5a63ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7bda2c0e354724a43506ff00be8eac, user_id=a5de257ac9db41c28ac793891661c1a1, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.363892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b3ee8-64da-4e65-8a2f-74b5b5a63ed6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.463953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1256/2527] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:39 2022] POST /identity/v3/users => generated 285 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:39.467921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b85e0-2b06-45a9-a062-47c250c397c8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.478102 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b85e0-2b06-45a9-a062-47c250c397c8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.483968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d41b3-7d9b-47b8-8f21-cae762f9e3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.484579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d41b3-7d9b-47b8-8f21-cae762f9e3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.484861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d41b3-7d9b-47b8-8f21-cae762f9e3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.485139 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d41b3-7d9b-47b8-8f21-cae762f9e3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.486560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d41b3-7d9b-47b8-8f21-cae762f9e3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.488680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d41b3-7d9b-47b8-8f21-cae762f9e3f2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.491125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:39.494811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1272/2528] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:39 2022] PUT /identity/v3/projects/dd7bda2c0e354724a43506ff00be8eac/users/a5de257ac9db41c28ac793891661c1a1/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:39.496069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b3ee8-64da-4e65-8a2f-74b5b5a63ed6 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.501054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1257/2529] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:39.504533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d41b3-7d9b-47b8-8f21-cae762f9e3f2 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.516041 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d41b3-7d9b-47b8-8f21-cae762f9e3f2 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.574238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b3ee8-64da-4e65-8a2f-74b5b5a63ed6 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.577814 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0a9a-763a-41f7-9766-0ba83059c03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.578491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0a9a-763a-41f7-9766-0ba83059c03b admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.578834 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0a9a-763a-41f7-9766-0ba83059c03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.579100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0a9a-763a-41f7-9766-0ba83059c03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.579783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e0a9a-763a-41f7-9766-0ba83059c03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.580157 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ff0ea-4dd1-4613-a76c-cb667a3ffab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.580766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ff0ea-4dd1-4613-a76c-cb667a3ffab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.581046 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ff0ea-4dd1-4613-a76c-cb667a3ffab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.581318 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ff0ea-4dd1-4613-a76c-cb667a3ffab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.581423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e0a9a-763a-41f7-9766-0ba83059c03b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.582003 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ff0ea-4dd1-4613-a76c-cb667a3ffab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.583579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ff0ea-4dd1-4613-a76c-cb667a3ffab5 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.592742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1258/2530] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:39.597332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1273/2531] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:39.597491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0a9a-763a-41f7-9766-0ba83059c03b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.598883 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ff0ea-4dd1-4613-a76c-cb667a3ffab5 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.608861 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0a9a-763a-41f7-9766-0ba83059c03b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.610739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ff0ea-4dd1-4613-a76c-cb667a3ffab5 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.615127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302176-c8f1-4589-9484-641298a1ca91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.615726 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c85e96-fdb7-4c99-89f8-afd5dc2895f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.616183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302176-c8f1-4589-9484-641298a1ca91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.616298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c85e96-fdb7-4c99-89f8-afd5dc2895f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.616492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c85e96-fdb7-4c99-89f8-afd5dc2895f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.616613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302176-c8f1-4589-9484-641298a1ca91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.616762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c85e96-fdb7-4c99-89f8-afd5dc2895f8 admin admin] PATH_INFO: `/v3/projects/dd7bda2c0e354724a43506ff00be8eac/users/a5de257ac9db41c28ac793891661c1a1/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.616865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302176-c8f1-4589-9484-641298a1ca91 admin admin] PATH_INFO: `/v3/projects/0e8b95165c674b428e62551d0bfdbb35/users/2209b18b12094469855b105418d02819/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.618855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c85e96-fdb7-4c99-89f8-afd5dc2895f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7bda2c0e354724a43506ff00be8eac, user_id=a5de257ac9db41c28ac793891661c1a1, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.621936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c85e96-fdb7-4c99-89f8-afd5dc2895f8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.632057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302176-c8f1-4589-9484-641298a1ca91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8b95165c674b428e62551d0bfdbb35, user_id=2209b18b12094469855b105418d02819, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.635655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1274/2532] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:39 2022] PUT /identity/v3/projects/dd7bda2c0e354724a43506ff00be8eac/users/a5de257ac9db41c28ac793891661c1a1/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:39.635838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302176-c8f1-4589-9484-641298a1ca91 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.644776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8c649-2d2f-48da-b31c-b03fa379c775 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.645088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8c649-2d2f-48da-b31c-b03fa379c775 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.645386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8c649-2d2f-48da-b31c-b03fa379c775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.676384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-08e8c649-2d2f-48da-b31c-b03fa379c775 None None] MFA Rules not processed for user `a5de257ac9db41c28ac793891661c1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:39.677601 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-08e8c649-2d2f-48da-b31c-b03fa379c775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:39.727781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:39.746966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1259/2533] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:39 2022] PUT /identity/v3/projects/0e8b95165c674b428e62551d0bfdbb35/users/2209b18b12094469855b105418d02819/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:39.751508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1275/2534] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:39.756900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba3ab-5687-47cd-abc1-dbe8587c9e56 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.757189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba3ab-5687-47cd-abc1-dbe8587c9e56 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.757466 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba3ab-5687-47cd-abc1-dbe8587c9e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.761515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8c649-2d2f-48da-b31c-b03fa379c775 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.763256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-08e8c649-2d2f-48da-b31c-b03fa379c775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:39.791629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-c87ba3ab-5687-47cd-abc1-dbe8587c9e56 None None] MFA Rules not processed for user `2209b18b12094469855b105418d02819`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:39.793257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c87ba3ab-5687-47cd-abc1-dbe8587c9e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:39.842629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8c649-2d2f-48da-b31c-b03fa379c775 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.880517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1260/2535] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:39.885709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ba3ab-5687-47cd-abc1-dbe8587c9e56 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:39.906519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668ee37-51a4-449a-9102-57b3a3ec488d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.907439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668ee37-51a4-449a-9102-57b3a3ec488d service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.908170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668ee37-51a4-449a-9102-57b3a3ec488d service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.908296 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668ee37-51a4-449a-9102-57b3a3ec488d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.956715 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ba3ab-5687-47cd-abc1-dbe8587c9e56 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:39.962030 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5070d-3372-4744-aa93-d1e0c9116f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:39.962642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5070d-3372-4744-aa93-d1e0c9116f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:39.962921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5070d-3372-4744-aa93-d1e0c9116f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:39.963198 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5070d-3372-4744-aa93-d1e0c9116f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:39.963826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5070d-3372-4744-aa93-d1e0c9116f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.966611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5070d-3372-4744-aa93-d1e0c9116f0e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:39.973829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:39.994657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668ee37-51a4-449a-9102-57b3a3ec488d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:39.996098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668ee37-51a4-449a-9102-57b3a3ec488d service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:40.006777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1276/2536] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:40.105054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1261/2537] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:39 2022] POST /identity/v3/projects => generated 369 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:40.109998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668ee37-51a4-449a-9102-57b3a3ec488d service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:40.121236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668ee37-51a4-449a-9102-57b3a3ec488d service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:40.179789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909e39f-d8d9-4173-9b52-cd5043f15283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:40.180409 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909e39f-d8d9-4173-9b52-cd5043f15283 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:40.180756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909e39f-d8d9-4173-9b52-cd5043f15283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:40.180951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909e39f-d8d9-4173-9b52-cd5043f15283 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:40.186033 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909e39f-d8d9-4173-9b52-cd5043f15283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:40.193380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909e39f-d8d9-4173-9b52-cd5043f15283 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:40.338417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1277/2538] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:40 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:40.343155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5070d-3372-4744-aa93-d1e0c9116f0e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:40.353748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5070d-3372-4744-aa93-d1e0c9116f0e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:40.359341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cf9dd-c888-446b-9b4c-456969406542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:40.359890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf9dd-c888-446b-9b4c-456969406542 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:40.360162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf9dd-c888-446b-9b4c-456969406542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:40.360428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf9dd-c888-446b-9b4c-456969406542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:40.361094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cf9dd-c888-446b-9b4c-456969406542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:40.362745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cf9dd-c888-446b-9b4c-456969406542 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:40.373568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1262/2539] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:40.393298 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909e39f-d8d9-4173-9b52-cd5043f15283 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:40.402258 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:40.408028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909e39f-d8d9-4173-9b52-cd5043f15283 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:40.413103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2d9e1-8e1f-4a14-8a2d-e48f8878d7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:40.413789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2d9e1-8e1f-4a14-8a2d-e48f8878d7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:40.414121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2d9e1-8e1f-4a14-8a2d-e48f8878d7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:40.414404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2d9e1-8e1f-4a14-8a2d-e48f8878d7cf admin admin] PATH_INFO: `/v3/projects/103b0f18a00743839dce45df8f6fabfa/users/0fbc12095db441ce907e9faf7b328470/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:40.431084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2d9e1-8e1f-4a14-8a2d-e48f8878d7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=103b0f18a00743839dce45df8f6fabfa, user_id=0fbc12095db441ce907e9faf7b328470, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:40.435881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2d9e1-8e1f-4a14-8a2d-e48f8878d7cf admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:40.530504 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1278/2540] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:40 2022] PUT /identity/v3/projects/103b0f18a00743839dce45df8f6fabfa/users/0fbc12095db441ce907e9faf7b328470/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:40.534360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cf9dd-c888-446b-9b4c-456969406542 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:40.546603 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cf9dd-c888-446b-9b4c-456969406542 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:40.614640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8973577-1d98-4910-893c-ca7d0d840fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:40.614640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8973577-1d98-4910-893c-ca7d0d840fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:40.614640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8973577-1d98-4910-893c-ca7d0d840fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:40.614640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8973577-1d98-4910-893c-ca7d0d840fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:40.615154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8973577-1d98-4910-893c-ca7d0d840fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:40.615154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8973577-1d98-4910-893c-ca7d0d840fd7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:40.624955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1263/2541] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:40.629804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2d9e1-8e1f-4a14-8a2d-e48f8878d7cf admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:40.640622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2d9e1-8e1f-4a14-8a2d-e48f8878d7cf admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:40.651572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:40.652250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:40.652531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:40.653336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] PATH_INFO: `/v3/projects/103b0f18a00743839dce45df8f6fabfa/users/0fbc12095db441ce907e9faf7b328470/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:40.655370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=103b0f18a00743839dce45df8f6fabfa, user_id=0fbc12095db441ce907e9faf7b328470, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:40.658038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:40.669671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1279/2542] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:40 2022] PUT /identity/v3/projects/103b0f18a00743839dce45df8f6fabfa/users/0fbc12095db441ce907e9faf7b328470/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:40.683242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d751-1236-40fe-b866-4c9306d0f0d8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:40.683543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d751-1236-40fe-b866-4c9306d0f0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:40.683830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d751-1236-40fe-b866-4c9306d0f0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:40.712888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-82d5d751-1236-40fe-b866-4c9306d0f0d8 None None] MFA Rules not processed for user `0fbc12095db441ce907e9faf7b328470`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:40.714162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-82d5d751-1236-40fe-b866-4c9306d0f0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:40.785864 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1264/2543] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:40.802411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:40.804405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:40.865733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f6c79-944b-4b08-855b-3bad2f47b48d admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:40.869350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:40.870041 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:40.870408 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:40.870683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:40.879784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:40.881268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:40.891721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1280/2544] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:41.149378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d751-1236-40fe-b866-4c9306d0f0d8 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:41.159959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d751-1236-40fe-b866-4c9306d0f0d8 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:41.163691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45eab69-5f88-4843-8b6a-392936464794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:41.164292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45eab69-5f88-4843-8b6a-392936464794 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:41.164581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45eab69-5f88-4843-8b6a-392936464794 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:41.164854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45eab69-5f88-4843-8b6a-392936464794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:41.217614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:41.220353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:41.226823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:41.231279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45eab69-5f88-4843-8b6a-392936464794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:41.232724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45eab69-5f88-4843-8b6a-392936464794 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:41.242720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1265/2545] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:41.305956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b61a2-25f3-4a8f-a312-82daa3023c06 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:41.310867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:41.311738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:41.311895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:41.312174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] PATH_INFO: `/v3/users/d37485f5cc7f4dd7bfdb0e4e6a846a13` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:41.313326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37485f5cc7f4dd7bfdb0e4e6a846a13)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:41.316187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:41.563457 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37485f5cc7f4dd7bfdb0e4e6a846a13'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:41.565390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:41.576292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37485f5cc7f4dd7bfdb0e4e6a846a13'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:41.657273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37485f5cc7f4dd7bfdb0e4e6a846a13'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:41.658463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37485f5cc7f4dd7bfdb0e4e6a846a13'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:41.659030 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37485f5cc7f4dd7bfdb0e4e6a846a13'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:41.683580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1281/2546] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:41 2022] DELETE /identity/v3/users/d37485f5cc7f4dd7bfdb0e4e6a846a13 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:41.685248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d43b-8177-4330-bf9f-72cfd52b4dfe None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:41.685628 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d43b-8177-4330-bf9f-72cfd52b4dfe None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:41.686019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d43b-8177-4330-bf9f-72cfd52b4dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:41.707430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a7d8d43b-8177-4330-bf9f-72cfd52b4dfe None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:41.708661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a7d8d43b-8177-4330-bf9f-72cfd52b4dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:41.769370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1266/2547] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:41.779846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:41.781254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:41.843017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5961-7b49-4b9a-9fcc-0c1be191762e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:41.854371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6f8a9-8629-4ea9-b404-76d0e34c58a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:41.856153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f8a9-8629-4ea9-b404-76d0e34c58a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:41.856153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f8a9-8629-4ea9-b404-76d0e34c58a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:41.856153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f8a9-8629-4ea9-b404-76d0e34c58a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:41.865461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6f8a9-8629-4ea9-b404-76d0e34c58a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:41.866957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6f8a9-8629-4ea9-b404-76d0e34c58a3 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:41.877640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1282/2548] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:42.332904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8d43b-8177-4330-bf9f-72cfd52b4dfe None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:42.334585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a7d8d43b-8177-4330-bf9f-72cfd52b4dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:42.364391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8d43b-8177-4330-bf9f-72cfd52b4dfe None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:42.369408 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:42.370084 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:42.370369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:42.370646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] PATH_INFO: `/v3/projects/10b541da08fa458b811cb0e8679d71ab` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:42.372345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b541da08fa458b811cb0e8679d71ab)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:42.374573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:42.504127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b541da08fa458b811cb0e8679d71ab', 'request_id': 'req-b404bad1-f603-4061-b609-fc38c179d796'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.557103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b541da08fa458b811cb0e8679d71ab', 'request_id': 'req-b404bad1-f603-4061-b609-fc38c179d796'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.563827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b541da08fa458b811cb0e8679d71ab', 'request_id': 'req-b404bad1-f603-4061-b609-fc38c179d796'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.575708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] The token cache is being invalidate because project 10b541da08fa458b811cb0e8679d71ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:42.576004 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.576669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.578205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1267/2549] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:42 2022] DELETE /identity/v3/projects/10b541da08fa458b811cb0e8679d71ab => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:42.582427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6f8a9-8629-4ea9-b404-76d0e34c58a3 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:42.593489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6f8a9-8629-4ea9-b404-76d0e34c58a3 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:42.662506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:42.663141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:42.663417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:42.663744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] PATH_INFO: `/v3/users/49d590c2adfe43df82303f321c3cc307` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:42.664927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49d590c2adfe43df82303f321c3cc307)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:42.667002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:42.854617 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49d590c2adfe43df82303f321c3cc307'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.869168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49d590c2adfe43df82303f321c3cc307'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.985926 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49d590c2adfe43df82303f321c3cc307'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.986781 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49d590c2adfe43df82303f321c3cc307'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.987535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49d590c2adfe43df82303f321c3cc307'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:42.989253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:43.007103 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1283/2550] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:42 2022] DELETE /identity/v3/users/49d590c2adfe43df82303f321c3cc307 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:43.500434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:43.506064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:43.518118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404bad1-f603-4061-b609-fc38c179d796 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:43.592652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:43.593794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:43.594098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:43.594375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] PATH_INFO: `/v3/projects/abccb3aed9cf4a3987b2ec78eef26b9e` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:43.595620 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=abccb3aed9cf4a3987b2ec78eef26b9e)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:43.597524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:43.626966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:43.720871 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:43.722353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:43.737097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e8987-caf9-482a-87e7-8eabab229111 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:43.753315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:43.753315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:43.753315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:43.753315 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] PATH_INFO: `/v3/users/e45bef4ec09c430395598bae9a0dd598` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:43.753749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45bef4ec09c430395598bae9a0dd598)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:43.753749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:43.801489 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abccb3aed9cf4a3987b2ec78eef26b9e', 'request_id': 'req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:43.920793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abccb3aed9cf4a3987b2ec78eef26b9e', 'request_id': 'req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:43.927725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abccb3aed9cf4a3987b2ec78eef26b9e', 'request_id': 'req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:43.941352 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] The token cache is being invalidate because project abccb3aed9cf4a3987b2ec78eef26b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:43.941654 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:43.942450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-fe8bbcb8-2759-4c84-a2f5-ff31002176cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:43.944060 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1268/2551] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:43 2022] DELETE /identity/v3/projects/abccb3aed9cf4a3987b2ec78eef26b9e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:43.949174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed07dcb-e049-406d-b311-2824120a2260 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:43.949718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed07dcb-e049-406d-b311-2824120a2260 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:43.950146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed07dcb-e049-406d-b311-2824120a2260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:43.969730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-fed07dcb-e049-406d-b311-2824120a2260 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:43.970965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-fed07dcb-e049-406d-b311-2824120a2260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:44.017733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45bef4ec09c430395598bae9a0dd598'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:44.030671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45bef4ec09c430395598bae9a0dd598'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:44.031818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1269/2552] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:44.040137 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6de1e-ef22-400a-a172-578169baf9a8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.045637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6de1e-ef22-400a-a172-578169baf9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.045637 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6de1e-ef22-400a-a172-578169baf9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.062910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-abb6de1e-ef22-400a-a172-578169baf9a8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:44.064195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-abb6de1e-ef22-400a-a172-578169baf9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:44.072369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1270/2553] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:44.076608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6de1e-ef22-400a-a172-578169baf9a8 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.085734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6de1e-ef22-400a-a172-578169baf9a8 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.090975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.091567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.091843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.092114 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.092756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:44.093057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.094802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:44.107251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1271/2554] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:44.118911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.129491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f811d24-f9d2-4058-8a70-a753ef46cb13 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.134247 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698b2de-5f88-4164-b929-71e33800f59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.134844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698b2de-5f88-4164-b929-71e33800f59f admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.135125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698b2de-5f88-4164-b929-71e33800f59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.135401 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698b2de-5f88-4164-b929-71e33800f59f admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.136019 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698b2de-5f88-4164-b929-71e33800f59f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.138213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698b2de-5f88-4164-b929-71e33800f59f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:44.146044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45bef4ec09c430395598bae9a0dd598'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:44.146406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45bef4ec09c430395598bae9a0dd598'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:44.146931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c4c6eb70-57d6-414f-9b41-0bfb2a4fafb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45bef4ec09c430395598bae9a0dd598'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:44.162975 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1284/2555] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:43 2022] DELETE /identity/v3/users/e45bef4ec09c430395598bae9a0dd598 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:44.167531 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbc08-da10-4c82-a9eb-899c609628ca None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.167851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbc08-da10-4c82-a9eb-899c609628ca None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.168851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbc08-da10-4c82-a9eb-899c609628ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.190109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-0c0dbc08-da10-4c82-a9eb-899c609628ca None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:44.191362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0c0dbc08-da10-4c82-a9eb-899c609628ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:44.254809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1285/2556] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:44.264976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dbc08-da10-4c82-a9eb-899c609628ca None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.266665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0c0dbc08-da10-4c82-a9eb-899c609628ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:44.331812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dbc08-da10-4c82-a9eb-899c609628ca None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.335632 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.336242 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.336515 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.336782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.347354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.348809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:44.361438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1286/2557] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:44.367765 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1272/2558] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:44 2022] POST /identity/v3/projects => generated 367 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:44.372916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.379844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:44.386892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:44.392076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1370-eb27-4ad2-a214-67131528e451 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.472113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fe5e2-3144-4dbb-b3c6-1ed783a0ee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.472113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fe5e2-3144-4dbb-b3c6-1ed783a0ee33 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.472113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fe5e2-3144-4dbb-b3c6-1ed783a0ee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.472113 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fe5e2-3144-4dbb-b3c6-1ed783a0ee33 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.472555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fe5e2-3144-4dbb-b3c6-1ed783a0ee33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.473977 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fe5e2-3144-4dbb-b3c6-1ed783a0ee33 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:44.570571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1287/2559] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:44 2022] POST /identity/v3/users => generated 284 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:44.576324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698b2de-5f88-4164-b929-71e33800f59f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.590214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698b2de-5f88-4164-b929-71e33800f59f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.596411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad5aeb-0991-4b6f-a87c-5855e682b862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.597135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad5aeb-0991-4b6f-a87c-5855e682b862 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.597486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad5aeb-0991-4b6f-a87c-5855e682b862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.597882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad5aeb-0991-4b6f-a87c-5855e682b862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.598646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad5aeb-0991-4b6f-a87c-5855e682b862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.600309 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad5aeb-0991-4b6f-a87c-5855e682b862 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:44.615118 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1273/2560] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:44.616163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fe5e2-3144-4dbb-b3c6-1ed783a0ee33 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.627734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fe5e2-3144-4dbb-b3c6-1ed783a0ee33 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.633843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184303-a2e6-4f3d-88ea-3a09fe0f2309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.634544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184303-a2e6-4f3d-88ea-3a09fe0f2309 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.634894 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184303-a2e6-4f3d-88ea-3a09fe0f2309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.635612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184303-a2e6-4f3d-88ea-3a09fe0f2309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.635980 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3184303-a2e6-4f3d-88ea-3a09fe0f2309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.638090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3184303-a2e6-4f3d-88ea-3a09fe0f2309 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:44.650345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1288/2561] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:44.655070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad5aeb-0991-4b6f-a87c-5855e682b862 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.665756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad5aeb-0991-4b6f-a87c-5855e682b862 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.670855 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.672794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.672794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.672794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] PATH_INFO: `/v3/projects/b2402696d3094e30a080535b7e9b4027/users/e0da0b71befd478b8be9063d8a339537/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.689213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2402696d3094e30a080535b7e9b4027, user_id=e0da0b71befd478b8be9063d8a339537, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.693107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:44.778389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1274/2562] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:44 2022] PUT /identity/v3/projects/b2402696d3094e30a080535b7e9b4027/users/e0da0b71befd478b8be9063d8a339537/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:44.789267 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e2a8-4526-4888-9de7-bc674e78c771 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.789812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e2a8-4526-4888-9de7-bc674e78c771 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.789812 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e2a8-4526-4888-9de7-bc674e78c771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.810593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.812682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:44.822769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-b672e2a8-4526-4888-9de7-bc674e78c771 None None] MFA Rules not processed for user `e0da0b71befd478b8be9063d8a339537`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:44.824013 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b672e2a8-4526-4888-9de7-bc674e78c771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:44.885589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d505318-6cb5-4f26-8259-7955d4cbaeec admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.893665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.894822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.895296 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.895772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] PATH_INFO: `/v3/projects/c58b2fa0b4e4476ea77070b96e03a82e` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.897809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58b2fa0b4e4476ea77070b96e03a82e)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.901532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:44.908454 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1289/2563] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:44.909544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e2a8-4526-4888-9de7-bc674e78c771 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:44.925739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e2a8-4526-4888-9de7-bc674e78c771 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:44.930746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01dc02-1786-4675-a346-6dc193c72978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:44.931354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01dc02-1786-4675-a346-6dc193c72978 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:44.931631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01dc02-1786-4675-a346-6dc193c72978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:44.931898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01dc02-1786-4675-a346-6dc193c72978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:44.932521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01dc02-1786-4675-a346-6dc193c72978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:44.935473 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01dc02-1786-4675-a346-6dc193c72978 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:45.038704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58b2fa0b4e4476ea77070b96e03a82e', 'request_id': 'req-d1f00e8b-ec9e-448d-949f-ee3701d4c734'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.058289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1290/2564] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:44 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:45.062869 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01dc02-1786-4675-a346-6dc193c72978 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:45.113417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58b2fa0b4e4476ea77070b96e03a82e', 'request_id': 'req-d1f00e8b-ec9e-448d-949f-ee3701d4c734'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.122334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58b2fa0b4e4476ea77070b96e03a82e', 'request_id': 'req-d1f00e8b-ec9e-448d-949f-ee3701d4c734'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.129793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01dc02-1786-4675-a346-6dc193c72978 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:45.134206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] The token cache is being invalidate because project c58b2fa0b4e4476ea77070b96e03a82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:45.134504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.135258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.135961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:45.136581 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:45.136884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:45.137175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:45.138082 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:45.138756 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1275/2565] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:44 2022] DELETE /identity/v3/projects/c58b2fa0b4e4476ea77070b96e03a82e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:45.140122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:45.144884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:45.146710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:45.225544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f00e8b-ec9e-448d-949f-ee3701d4c734 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:45.230396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:45.231045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:45.231324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:45.231602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] PATH_INFO: `/v3/users/a5de257ac9db41c28ac793891661c1a1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:45.232888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5de257ac9db41c28ac793891661c1a1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:45.234879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:45.244415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1291/2566] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:45 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:45.249492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:45.256807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:45.258657 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:45.272375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af99c4-823b-4b09-84ac-bdbfbf49b34e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:45.277385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710cf2-5260-4451-b195-80125d0e0273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:45.279807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710cf2-5260-4451-b195-80125d0e0273 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:45.280121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710cf2-5260-4451-b195-80125d0e0273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:45.280493 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710cf2-5260-4451-b195-80125d0e0273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:45.281174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f710cf2-5260-4451-b195-80125d0e0273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:45.282799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f710cf2-5260-4451-b195-80125d0e0273 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:45.295250 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1292/2567] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:45.298953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710cf2-5260-4451-b195-80125d0e0273 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:45.309237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710cf2-5260-4451-b195-80125d0e0273 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:45.314226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cd3c5-3f40-4637-bdda-f3fbb18cad72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:45.314887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cd3c5-3f40-4637-bdda-f3fbb18cad72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:45.315174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cd3c5-3f40-4637-bdda-f3fbb18cad72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:45.315453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cd3c5-3f40-4637-bdda-f3fbb18cad72 admin admin] PATH_INFO: `/v3/projects/626d25a7152a403586ef87a9d29d7e75/users/c81eae8834374ac68506cbc462c04946/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:45.335275 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cd3c5-3f40-4637-bdda-f3fbb18cad72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=626d25a7152a403586ef87a9d29d7e75, user_id=c81eae8834374ac68506cbc462c04946, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:45.338222 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cd3c5-3f40-4637-bdda-f3fbb18cad72 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:45.454040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5de257ac9db41c28ac793891661c1a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.487131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1293/2568] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:45 2022] PUT /identity/v3/projects/626d25a7152a403586ef87a9d29d7e75/users/c81eae8834374ac68506cbc462c04946/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:45.491743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cd3c5-3f40-4637-bdda-f3fbb18cad72 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:45.571568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cd3c5-3f40-4637-bdda-f3fbb18cad72 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:45.585825 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5de257ac9db41c28ac793891661c1a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.590372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:45.590731 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5de257ac9db41c28ac793891661c1a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.590818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:45.590818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:45.590818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:45.590818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:45.590818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:45.595736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5de257ac9db41c28ac793891661c1a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.596900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-52b09339-a2a0-412b-a623-182f5684f44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5de257ac9db41c28ac793891661c1a1'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:45.602939 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1294/2569] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:45.608095 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:45.610228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:45.624393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae2cb1-c52d-43c7-8198-e766ba8d4909 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:45.625757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1276/2570] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:45 2022] DELETE /identity/v3/users/a5de257ac9db41c28ac793891661c1a1 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:45.635843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:45.708458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:45.709135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:45.709418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:45.709696 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] PATH_INFO: `/v3/projects/626d25a7152a403586ef87a9d29d7e75/users/c81eae8834374ac68506cbc462c04946/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:45.711827 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] RBAC: Authorizing `identity:create_grant(project_id=626d25a7152a403586ef87a9d29d7e75, user_id=c81eae8834374ac68506cbc462c04946, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:45.714613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:45.726174 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1295/2571] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:45 2022] PUT /identity/v3/projects/626d25a7152a403586ef87a9d29d7e75/users/c81eae8834374ac68506cbc462c04946/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:45.738578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e0f2-7370-4870-9d2d-b40889cd29f0 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:45.739112 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e0f2-7370-4870-9d2d-b40889cd29f0 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:45.739392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e0f2-7370-4870-9d2d-b40889cd29f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:45.772788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-3647e0f2-7370-4870-9d2d-b40889cd29f0 None None] MFA Rules not processed for user `c81eae8834374ac68506cbc462c04946`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:45.775808 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3647e0f2-7370-4870-9d2d-b40889cd29f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:45.859369 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1277/2572] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:45.872466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:45.874536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:45.956844 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64b420-d89a-4360-aa82-7d5db88c821f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:45.960706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a7cd3-7153-454c-ac66-85ae605e86dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:45.961316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7cd3-7153-454c-ac66-85ae605e86dd service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:45.961594 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7cd3-7153-454c-ac66-85ae605e86dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:45.961890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7cd3-7153-454c-ac66-85ae605e86dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:45.972317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a7cd3-7153-454c-ac66-85ae605e86dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:45.974118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a7cd3-7153-454c-ac66-85ae605e86dd service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:45.985787 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1296/2573] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:46.154458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647e0f2-7370-4870-9d2d-b40889cd29f0 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:46.155933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3647e0f2-7370-4870-9d2d-b40889cd29f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:46.192598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:46.211500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a7cd3-7153-454c-ac66-85ae605e86dd service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:46.221456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a7cd3-7153-454c-ac66-85ae605e86dd service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:46.225550 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012284a-8b87-4853-b139-af963b57de0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:46.226204 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012284a-8b87-4853-b139-af963b57de0b service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:46.226488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012284a-8b87-4853-b139-af963b57de0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:46.226764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012284a-8b87-4853-b139-af963b57de0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:46.228571 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5012284a-8b87-4853-b139-af963b57de0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:46.240107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647e0f2-7370-4870-9d2d-b40889cd29f0 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:46.246400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:46.248106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:46.248106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:46.248362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] PATH_INFO: `/v3/projects/dd7bda2c0e354724a43506ff00be8eac` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:46.250261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7bda2c0e354724a43506ff00be8eac)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:46.257099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:46.257224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:46.302367 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012284a-8b87-4853-b139-af963b57de0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:46.303834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012284a-8b87-4853-b139-af963b57de0b service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:46.314004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1297/2574] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:46.372256 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7bda2c0e354724a43506ff00be8eac', 'request_id': 'req-04736b9e-253f-475a-b106-3c883a58c33d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:46.471997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7bda2c0e354724a43506ff00be8eac', 'request_id': 'req-04736b9e-253f-475a-b106-3c883a58c33d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:46.479794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7bda2c0e354724a43506ff00be8eac', 'request_id': 'req-04736b9e-253f-475a-b106-3c883a58c33d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:46.492031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] The token cache is being invalidate because project dd7bda2c0e354724a43506ff00be8eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:46.492324 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:46.492982 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-04736b9e-253f-475a-b106-3c883a58c33d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:46.494514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1278/2575] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:46 2022] DELETE /identity/v3/projects/dd7bda2c0e354724a43506ff00be8eac => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:46.501294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bc50-92c6-491c-8696-6abc2b24a800 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:46.501566 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bc50-92c6-491c-8696-6abc2b24a800 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:46.501845 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bc50-92c6-491c-8696-6abc2b24a800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:46.523138 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-bdd2bc50-92c6-491c-8696-6abc2b24a800 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:46.524426 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bdd2bc50-92c6-491c-8696-6abc2b24a800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:46.589742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1298/2576] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:46.598866 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328821-a7db-4fb3-a771-341205d81b3e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:46.599430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328821-a7db-4fb3-a771-341205d81b3e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:46.599430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328821-a7db-4fb3-a771-341205d81b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:46.619384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6e328821-a7db-4fb3-a771-341205d81b3e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:46.620600 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6e328821-a7db-4fb3-a771-341205d81b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:46.629197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1279/2577] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:46.633556 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bc50-92c6-491c-8696-6abc2b24a800 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:46.644139 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bc50-92c6-491c-8696-6abc2b24a800 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:46.648683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:46.649285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:46.649576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:46.649829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:46.650589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:46.650806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:46.652379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:46.664118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1299/2578] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:46.669820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e328821-a7db-4fb3-a771-341205d81b3e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:46.680362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e328821-a7db-4fb3-a771-341205d81b3e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:46.686161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:46.686761 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:46.687058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:46.687306 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:46.687945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:46.689891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:46.854827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1280/2579] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:46 2022] POST /identity/v3/projects => generated 361 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:46.858205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:46.873263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5530669-c07e-4d33-9782-5bc09c5245c2 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:46.936349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddecef-e994-4ac3-9af1-dbcf8e6d7250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:46.936973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddecef-e994-4ac3-9af1-dbcf8e6d7250 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:46.937259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddecef-e994-4ac3-9af1-dbcf8e6d7250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:46.937535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddecef-e994-4ac3-9af1-dbcf8e6d7250 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:46.938388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddecef-e994-4ac3-9af1-dbcf8e6d7250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:46.940404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddecef-e994-4ac3-9af1-dbcf8e6d7250 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:46.950718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:47.039613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:47.041460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:47.051832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:47.057526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814ad2c-f2ec-4aa9-b7d6-f6d225390d82 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:47.063462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:47.064241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.066957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.066957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] PATH_INFO: `/v3/users/2209b18b12094469855b105418d02819` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.067148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1300/2580] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:46 2022] POST /identity/v3/users => generated 281 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:47.070525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2209b18b12094469855b105418d02819)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:47.070525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:47.072757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddecef-e994-4ac3-9af1-dbcf8e6d7250 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:47.084951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddecef-e994-4ac3-9af1-dbcf8e6d7250 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:47.089669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5666bb9-9558-4b1c-a64a-65b9b6439f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:47.090843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5666bb9-9558-4b1c-a64a-65b9b6439f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.091133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5666bb9-9558-4b1c-a64a-65b9b6439f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.091684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5666bb9-9558-4b1c-a64a-65b9b6439f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.093091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5666bb9-9558-4b1c-a64a-65b9b6439f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:47.094865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5666bb9-9558-4b1c-a64a-65b9b6439f2b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:47.108887 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1301/2581] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:47.113293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5666bb9-9558-4b1c-a64a-65b9b6439f2b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:47.123586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5666bb9-9558-4b1c-a64a-65b9b6439f2b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:47.128406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98825b7e-e88b-4216-9f70-02eb3a9fecdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:47.129026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825b7e-e88b-4216-9f70-02eb3a9fecdf admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.129306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825b7e-e88b-4216-9f70-02eb3a9fecdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.129582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825b7e-e88b-4216-9f70-02eb3a9fecdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.130292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98825b7e-e88b-4216-9f70-02eb3a9fecdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:47.131868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98825b7e-e88b-4216-9f70-02eb3a9fecdf admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:47.142487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1302/2582] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:47.149037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98825b7e-e88b-4216-9f70-02eb3a9fecdf admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:47.159249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98825b7e-e88b-4216-9f70-02eb3a9fecdf admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:47.163948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a405ba-cf58-465a-b8a2-f4d8bb224123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:47.164575 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a405ba-cf58-465a-b8a2-f4d8bb224123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.164857 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a405ba-cf58-465a-b8a2-f4d8bb224123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.165132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a405ba-cf58-465a-b8a2-f4d8bb224123 admin admin] PATH_INFO: `/v3/projects/8bdb2126c4a9494c87ccb10596ee3273/users/64c38265005e465e9c7ded53b4b632ce/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.180730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a405ba-cf58-465a-b8a2-f4d8bb224123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdb2126c4a9494c87ccb10596ee3273, user_id=64c38265005e465e9c7ded53b4b632ce, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:47.183403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a405ba-cf58-465a-b8a2-f4d8bb224123 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:47.257579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1303/2583] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:47 2022] PUT /identity/v3/projects/8bdb2126c4a9494c87ccb10596ee3273/users/64c38265005e465e9c7ded53b4b632ce/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:47.268644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07160ea2-98c3-4427-a1bc-57e98b52be9e None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.268934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07160ea2-98c3-4427-a1bc-57e98b52be9e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.269213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07160ea2-98c3-4427-a1bc-57e98b52be9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.298342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-07160ea2-98c3-4427-a1bc-57e98b52be9e None None] MFA Rules not processed for user `64c38265005e465e9c7ded53b4b632ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:47.301155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-07160ea2-98c3-4427-a1bc-57e98b52be9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:47.305619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2209b18b12094469855b105418d02819'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.365765 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1304/2584] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:47.406346 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2209b18b12094469855b105418d02819'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.407034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2209b18b12094469855b105418d02819'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.412916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07160ea2-98c3-4427-a1bc-57e98b52be9e None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:47.414400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-07160ea2-98c3-4427-a1bc-57e98b52be9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:47.414956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2209b18b12094469855b105418d02819'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.415627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-26559a49-38dd-469c-90ab-847220138e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2209b18b12094469855b105418d02819'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.434881 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1281/2585] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:47 2022] DELETE /identity/v3/users/2209b18b12094469855b105418d02819 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:47.439249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e962d1-3045-40e0-920a-85128125971c None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.439613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e962d1-3045-40e0-920a-85128125971c None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.439946 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e962d1-3045-40e0-920a-85128125971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.459050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-91e962d1-3045-40e0-920a-85128125971c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:47.460271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-91e962d1-3045-40e0-920a-85128125971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:47.474528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07160ea2-98c3-4427-a1bc-57e98b52be9e None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:47.523036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1282/2586] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:47.525159 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:47.525772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.526206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.526476 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.530933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:47.533246 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e962d1-3045-40e0-920a-85128125971c None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:47.534698 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-91e962d1-3045-40e0-920a-85128125971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:47.550865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:47.592927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:47.594418 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:47.594785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e962d1-3045-40e0-920a-85128125971c None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:47.598811 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872034e0-ad9c-4736-93f0-9d7886ac40e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:47.599501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872034e0-ad9c-4736-93f0-9d7886ac40e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.600052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872034e0-ad9c-4736-93f0-9d7886ac40e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.600161 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872034e0-ad9c-4736-93f0-9d7886ac40e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.604333 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1305/2587] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:47.611692 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872034e0-ad9c-4736-93f0-9d7886ac40e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:47.612433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872034e0-ad9c-4736-93f0-9d7886ac40e7 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:47.620439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:47.622397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:47.623943 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1283/2588] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:47.631950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:47.641764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98bda69-b35c-48a1-8d3a-0c40e41d60d3 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:47.642621 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:47.643341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.643772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.644123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] PATH_INFO: `/v3/users/64c38265005e465e9c7ded53b4b632ce` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:47.645754 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c38265005e465e9c7ded53b4b632ce)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:47.647791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:47.865221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c38265005e465e9c7ded53b4b632ce'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.880366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c38265005e465e9c7ded53b4b632ce'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.968269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c38265005e465e9c7ded53b4b632ce'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.968901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c38265005e465e9c7ded53b4b632ce'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.971218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c38265005e465e9c7ded53b4b632ce'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:47.985911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1306/2589] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:47 2022] DELETE /identity/v3/users/64c38265005e465e9c7ded53b4b632ce => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:47.990615 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46350c-4efc-45bd-b312-53e2b28c3dbe None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:47.990904 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46350c-4efc-45bd-b312-53e2b28c3dbe None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:47.991180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46350c-4efc-45bd-b312-53e2b28c3dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:48.011262 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0a46350c-4efc-45bd-b312-53e2b28c3dbe None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:48.012514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0a46350c-4efc-45bd-b312-53e2b28c3dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:48.055847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:48.061400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:48.079490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1284/2590] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:48.092151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46350c-4efc-45bd-b312-53e2b28c3dbe None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:48.093714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0a46350c-4efc-45bd-b312-53e2b28c3dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:48.145434 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c9fc0-fb0c-40dd-b67c-1130ee000bdb admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:48.150683 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:48.151936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:48.152237 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:48.156956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] PATH_INFO: `/v3/projects/0e8b95165c674b428e62551d0bfdbb35` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:48.156956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8b95165c674b428e62551d0bfdbb35)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:48.156956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:48.158414 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46350c-4efc-45bd-b312-53e2b28c3dbe None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:48.162299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4a1b0-7424-4a74-b44e-e6eaffde64b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:48.162910 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4a1b0-7424-4a74-b44e-e6eaffde64b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:48.163189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4a1b0-7424-4a74-b44e-e6eaffde64b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:48.163563 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4a1b0-7424-4a74-b44e-e6eaffde64b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:48.174363 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4a1b0-7424-4a74-b44e-e6eaffde64b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:48.181207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4a1b0-7424-4a74-b44e-e6eaffde64b8 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:48.185314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:48.190344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1285/2591] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:48.292653 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8b95165c674b428e62551d0bfdbb35', 'request_id': 'req-c845fc05-f133-4661-94f8-53e98f51e8eb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.366573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8b95165c674b428e62551d0bfdbb35', 'request_id': 'req-c845fc05-f133-4661-94f8-53e98f51e8eb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.373669 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8b95165c674b428e62551d0bfdbb35', 'request_id': 'req-c845fc05-f133-4661-94f8-53e98f51e8eb'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.385698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] The token cache is being invalidate because project 0e8b95165c674b428e62551d0bfdbb35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:48.385698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.386392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.389153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1307/2592] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:48 2022] DELETE /identity/v3/projects/0e8b95165c674b428e62551d0bfdbb35 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:48.392829 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4a1b0-7424-4a74-b44e-e6eaffde64b8 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:48.403527 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4a1b0-7424-4a74-b44e-e6eaffde64b8 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:48.468535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:48.468977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:48.469265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:48.469608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] PATH_INFO: `/v3/users/0fbc12095db441ce907e9faf7b328470` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:48.470822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbc12095db441ce907e9faf7b328470)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:48.472950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:48.562278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:48.595094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:48.597506 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:48.642931 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845fc05-f133-4661-94f8-53e98f51e8eb admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:48.649168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:48.650229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:48.650698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:48.651073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] PATH_INFO: `/v3/projects/8bdb2126c4a9494c87ccb10596ee3273` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:48.652732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bdb2126c4a9494c87ccb10596ee3273)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:48.655304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:48.664532 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbc12095db441ce907e9faf7b328470'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.797055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbc12095db441ce907e9faf7b328470'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.797707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbc12095db441ce907e9faf7b328470'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.805314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbc12095db441ce907e9faf7b328470'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.805975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7291f33f-d057-460e-b4c6-3bf80cc1cacf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbc12095db441ce907e9faf7b328470'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.824076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1286/2593] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:48 2022] DELETE /identity/v3/users/0fbc12095db441ce907e9faf7b328470 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:48.869950 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bdb2126c4a9494c87ccb10596ee3273', 'request_id': 'req-bf264ca0-8f71-4b91-bea7-56cad71decde'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.937774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bdb2126c4a9494c87ccb10596ee3273', 'request_id': 'req-bf264ca0-8f71-4b91-bea7-56cad71decde'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.944807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bdb2126c4a9494c87ccb10596ee3273', 'request_id': 'req-bf264ca0-8f71-4b91-bea7-56cad71decde'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.955600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] The token cache is being invalidate because project 8bdb2126c4a9494c87ccb10596ee3273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:48.955890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.956602 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-bf264ca0-8f71-4b91-bea7-56cad71decde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:48.958223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1308/2594] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:48 2022] DELETE /identity/v3/projects/8bdb2126c4a9494c87ccb10596ee3273 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:48.963176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4368df-94bd-4a9c-b0d5-c219b1c5085e None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:48.963530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4368df-94bd-4a9c-b0d5-c219b1c5085e None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:48.963839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4368df-94bd-4a9c-b0d5-c219b1c5085e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:48.986315 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-2f4368df-94bd-4a9c-b0d5-c219b1c5085e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:48.987540 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2f4368df-94bd-4a9c-b0d5-c219b1c5085e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:49.052762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1287/2595] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:49.062206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a1468-b0e4-44d8-bab9-2054a742acea None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.062596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a1468-b0e4-44d8-bab9-2054a742acea None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.062943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a1468-b0e4-44d8-bab9-2054a742acea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.084586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d41a1468-b0e4-44d8-bab9-2054a742acea None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:49.085813 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d41a1468-b0e4-44d8-bab9-2054a742acea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:49.094614 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1309/2596] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:49.098947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4368df-94bd-4a9c-b0d5-c219b1c5085e None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.109226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4368df-94bd-4a9c-b0d5-c219b1c5085e None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.114024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.114374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.114665 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.114942 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.115678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:49.116203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.117610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.133781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1288/2597] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:49.134284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a1468-b0e4-44d8-bab9-2054a742acea None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.144632 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a1468-b0e4-44d8-bab9-2054a742acea None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.149428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.150303 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.150591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.150868 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.151513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.154085 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.258728 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1310/2598] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:49 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:49.263558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.273938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea592457-27c7-4ce1-9906-e2ac552dc858 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.306523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.309840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:49.338200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581c212-bd4e-474d-8f00-0b3b102ef347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.338200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581c212-bd4e-474d-8f00-0b3b102ef347 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.338200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581c212-bd4e-474d-8f00-0b3b102ef347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.338200 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581c212-bd4e-474d-8f00-0b3b102ef347 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.338973 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581c212-bd4e-474d-8f00-0b3b102ef347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.340421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581c212-bd4e-474d-8f00-0b3b102ef347 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.383351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e378d0-7824-4e0b-8102-0aadf1b7247c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.388782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.389403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.389680 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.389983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] PATH_INFO: `/v3/projects/103b0f18a00743839dce45df8f6fabfa` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.391375 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=103b0f18a00743839dce45df8f6fabfa)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.393386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.399875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:49.417154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1289/2599] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:49 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:49.421244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581c212-bd4e-474d-8f00-0b3b102ef347 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.431171 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581c212-bd4e-474d-8f00-0b3b102ef347 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.436622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe75ad-795d-431c-bd44-dcc9538d13c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.437219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe75ad-795d-431c-bd44-dcc9538d13c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.437496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe75ad-795d-431c-bd44-dcc9538d13c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.437762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe75ad-795d-431c-bd44-dcc9538d13c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.438437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe75ad-795d-431c-bd44-dcc9538d13c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.440099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe75ad-795d-431c-bd44-dcc9538d13c7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.452110 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1290/2600] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:49.455971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe75ad-795d-431c-bd44-dcc9538d13c7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.465786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe75ad-795d-431c-bd44-dcc9538d13c7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.470660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9609-9b53-4dbb-8a46-0481f8e5f629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.472205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9609-9b53-4dbb-8a46-0481f8e5f629 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.472205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9609-9b53-4dbb-8a46-0481f8e5f629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.472205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9609-9b53-4dbb-8a46-0481f8e5f629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.472580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce9609-9b53-4dbb-8a46-0481f8e5f629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.473975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce9609-9b53-4dbb-8a46-0481f8e5f629 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.479204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:49.484616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1291/2601] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:49.488700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9609-9b53-4dbb-8a46-0481f8e5f629 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.499767 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9609-9b53-4dbb-8a46-0481f8e5f629 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.510648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '103b0f18a00743839dce45df8f6fabfa', 'request_id': 'req-25f6c90a-78d2-4660-b3ea-6a60cb499948'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:49.555927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ed6cc-798c-493a-8dad-d280f950ae2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.556590 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed6cc-798c-493a-8dad-d280f950ae2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.556868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed6cc-798c-493a-8dad-d280f950ae2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.557386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed6cc-798c-493a-8dad-d280f950ae2a admin admin] PATH_INFO: `/v3/projects/06d78afb0fa541f996a8446bd684dbf4/users/2623b90df9c142858e88d30da1826f87/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.572845 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ed6cc-798c-493a-8dad-d280f950ae2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d78afb0fa541f996a8446bd684dbf4, user_id=2623b90df9c142858e88d30da1826f87, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.576585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ed6cc-798c-493a-8dad-d280f950ae2a admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.585535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '103b0f18a00743839dce45df8f6fabfa', 'request_id': 'req-25f6c90a-78d2-4660-b3ea-6a60cb499948'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:49.592947 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '103b0f18a00743839dce45df8f6fabfa', 'request_id': 'req-25f6c90a-78d2-4660-b3ea-6a60cb499948'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:49.603986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] The token cache is being invalidate because project 103b0f18a00743839dce45df8f6fabfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:49.604479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:49.609982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-25f6c90a-78d2-4660-b3ea-6a60cb499948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:49.609982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1311/2602] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:49 2022] DELETE /identity/v3/projects/103b0f18a00743839dce45df8f6fabfa => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:49.613054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfb668-8897-4027-94ec-4cbb51016895 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.613339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfb668-8897-4027-94ec-4cbb51016895 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.613616 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfb668-8897-4027-94ec-4cbb51016895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.634285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ffdfb668-8897-4027-94ec-4cbb51016895 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:49.635478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ffdfb668-8897-4027-94ec-4cbb51016895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:49.644597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1312/2603] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:49.652132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf3a7-5c37-42a4-8b40-ff9d4e889e56 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.652417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf3a7-5c37-42a4-8b40-ff9d4e889e56 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.652768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf3a7-5c37-42a4-8b40-ff9d4e889e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.673024 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4d1cf3a7-5c37-42a4-8b40-ff9d4e889e56 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:49.674249 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4d1cf3a7-5c37-42a4-8b40-ff9d4e889e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:49.682032 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1313/2604] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:49.687287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1cf3a7-5c37-42a4-8b40-ff9d4e889e56 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.690189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1292/2605] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:49 2022] PUT /identity/v3/projects/06d78afb0fa541f996a8446bd684dbf4/users/2623b90df9c142858e88d30da1826f87/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:49.696893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1cf3a7-5c37-42a4-8b40-ff9d4e889e56 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.700982 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620be30-ba4e-4c72-ab15-88d41ad19f34 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.702101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620be30-ba4e-4c72-ab15-88d41ad19f34 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.702556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620be30-ba4e-4c72-ab15-88d41ad19f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.734155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6620be30-ba4e-4c72-ab15-88d41ad19f34 None None] MFA Rules not processed for user `2623b90df9c142858e88d30da1826f87`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:49.735395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6620be30-ba4e-4c72-ab15-88d41ad19f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:49.780224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.780849 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.781128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.781402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.782472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:49.782777 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.784710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.798852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1314/2606] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:56:49.803299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.814832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12841f44-15e6-45ae-9d4a-85dfd737c5ec admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.820078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1293/2607] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:49.825219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5184a8d-9cb2-4273-bdc9-d61cdd9de973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.826081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5184a8d-9cb2-4273-bdc9-d61cdd9de973 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.826703 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5184a8d-9cb2-4273-bdc9-d61cdd9de973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.827093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5184a8d-9cb2-4273-bdc9-d61cdd9de973 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.827806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5184a8d-9cb2-4273-bdc9-d61cdd9de973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.828419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620be30-ba4e-4c72-ab15-88d41ad19f34 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.829873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6620be30-ba4e-4c72-ab15-88d41ad19f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:49.830380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5184a8d-9cb2-4273-bdc9-d61cdd9de973 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.892508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620be30-ba4e-4c72-ab15-88d41ad19f34 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.896228 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080eb83-847d-4bc3-8679-1015017afa88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:49.896853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080eb83-847d-4bc3-8679-1015017afa88 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:49.897123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080eb83-847d-4bc3-8679-1015017afa88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:49.897399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080eb83-847d-4bc3-8679-1015017afa88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:49.906709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080eb83-847d-4bc3-8679-1015017afa88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:49.908286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080eb83-847d-4bc3-8679-1015017afa88 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:49.919094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1294/2608] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:49.953842 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1315/2609] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:49 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:49.959750 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080eb83-847d-4bc3-8679-1015017afa88 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:49.973147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080eb83-847d-4bc3-8679-1015017afa88 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:49.994038 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:50.042481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4b74b-6402-4dff-b56e-7171f34d1637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:50.043203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b74b-6402-4dff-b56e-7171f34d1637 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:50.044132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b74b-6402-4dff-b56e-7171f34d1637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:50.044616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b74b-6402-4dff-b56e-7171f34d1637 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:50.045398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4b74b-6402-4dff-b56e-7171f34d1637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:50.047492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4b74b-6402-4dff-b56e-7171f34d1637 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:50.146224 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1295/2610] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:49 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:50.150364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5184a8d-9cb2-4273-bdc9-d61cdd9de973 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:50.162503 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5184a8d-9cb2-4273-bdc9-d61cdd9de973 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:50.177273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477f5f1-41d1-4e34-b0dd-59aae6704a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:50.177273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477f5f1-41d1-4e34-b0dd-59aae6704a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:50.177273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477f5f1-41d1-4e34-b0dd-59aae6704a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:50.177273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477f5f1-41d1-4e34-b0dd-59aae6704a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:50.178648 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477f5f1-41d1-4e34-b0dd-59aae6704a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:50.180583 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477f5f1-41d1-4e34-b0dd-59aae6704a9a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:50.191706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1316/2611] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:50.195956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4b74b-6402-4dff-b56e-7171f34d1637 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:50.210564 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4b74b-6402-4dff-b56e-7171f34d1637 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:50.213824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90a6c1-e47a-4e5e-9ccf-353192036767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:50.214508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90a6c1-e47a-4e5e-9ccf-353192036767 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:50.214797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90a6c1-e47a-4e5e-9ccf-353192036767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:50.215088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90a6c1-e47a-4e5e-9ccf-353192036767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:50.216438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90a6c1-e47a-4e5e-9ccf-353192036767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:50.218087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90a6c1-e47a-4e5e-9ccf-353192036767 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:50.228882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1296/2612] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:50.233679 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477f5f1-41d1-4e34-b0dd-59aae6704a9a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:50.244211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477f5f1-41d1-4e34-b0dd-59aae6704a9a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:50.249579 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:50.250272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:50.250557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:50.250835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] PATH_INFO: `/v3/projects/2eefb400a71e45c4996f6894680f1d87/users/3aa2b413880743fdb5bdd8250a7e217b/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:50.266656 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eefb400a71e45c4996f6894680f1d87, user_id=3aa2b413880743fdb5bdd8250a7e217b, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:50.269347 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:50.370909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1317/2613] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:50 2022] PUT /identity/v3/projects/2eefb400a71e45c4996f6894680f1d87/users/3aa2b413880743fdb5bdd8250a7e217b/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:50.380423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89a87-a5ec-4db5-b663-b68c24b23596 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:50.380718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89a87-a5ec-4db5-b663-b68c24b23596 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:50.380989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89a87-a5ec-4db5-b663-b68c24b23596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:50.417186 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ace89a87-a5ec-4db5-b663-b68c24b23596 None None] MFA Rules not processed for user `3aa2b413880743fdb5bdd8250a7e217b`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:50.418503 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ace89a87-a5ec-4db5-b663-b68c24b23596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:50.486565 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1297/2614] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:50.550837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:50.552585 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:50.573189 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:50.612767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55414257-ca0c-4b5e-8717-38d024088c7e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:50.616384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7533f6-443a-4568-af20-23b2c06c9ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:50.616990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7533f6-443a-4568-af20-23b2c06c9ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:50.617260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7533f6-443a-4568-af20-23b2c06c9ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:50.617536 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7533f6-443a-4568-af20-23b2c06c9ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:50.626596 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7533f6-443a-4568-af20-23b2c06c9ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:50.636231 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7533f6-443a-4568-af20-23b2c06c9ff0 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:50.636712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1318/2615] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:50.660271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace89a87-a5ec-4db5-b663-b68c24b23596 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:50.747406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace89a87-a5ec-4db5-b663-b68c24b23596 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:50.754348 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:50.754987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:50.755270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:50.760583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] PATH_INFO: `/v3/users/3aa2b413880743fdb5bdd8250a7e217b` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:50.760583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa2b413880743fdb5bdd8250a7e217b)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:50.760583 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:50.852443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:50.984232 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa2b413880743fdb5bdd8250a7e217b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:51.108124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa2b413880743fdb5bdd8250a7e217b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:51.108759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa2b413880743fdb5bdd8250a7e217b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:51.116138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa2b413880743fdb5bdd8250a7e217b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:51.116924 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa2b413880743fdb5bdd8250a7e217b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:51.134631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1298/2616] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:50 2022] DELETE /identity/v3/users/3aa2b413880743fdb5bdd8250a7e217b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:51.139990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9e25-2fb8-4e40-8576-6aa40efc71c8 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:51.140521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9e25-2fb8-4e40-8576-6aa40efc71c8 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:51.140626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9e25-2fb8-4e40-8576-6aa40efc71c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:51.160455 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4add9e25-2fb8-4e40-8576-6aa40efc71c8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:51.161722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4add9e25-2fb8-4e40-8576-6aa40efc71c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:51.229706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1319/2617] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:51.240356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:51.241933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:51.305463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81bf9e-886c-4dfe-bfa6-7b482916088c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:51.309814 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee492be3-bda8-478f-9076-f4be3960ce7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:51.310473 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492be3-bda8-478f-9076-f4be3960ce7b service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:51.310749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492be3-bda8-478f-9076-f4be3960ce7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:51.311020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492be3-bda8-478f-9076-f4be3960ce7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:51.320085 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee492be3-bda8-478f-9076-f4be3960ce7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:51.321545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee492be3-bda8-478f-9076-f4be3960ce7b service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:51.331787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1299/2618] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:51.773383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add9e25-2fb8-4e40-8576-6aa40efc71c8 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:51.775564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4add9e25-2fb8-4e40-8576-6aa40efc71c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:51.801023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add9e25-2fb8-4e40-8576-6aa40efc71c8 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:51.807663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:51.807663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:51.807663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:51.807663 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] PATH_INFO: `/v3/projects/2eefb400a71e45c4996f6894680f1d87` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:51.809313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eefb400a71e45c4996f6894680f1d87)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:51.811122 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:51.946410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eefb400a71e45c4996f6894680f1d87', 'request_id': 'req-1204d567-5c10-4706-a104-82ca92d08551'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:52.037202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eefb400a71e45c4996f6894680f1d87', 'request_id': 'req-1204d567-5c10-4706-a104-82ca92d08551'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:52.046391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eefb400a71e45c4996f6894680f1d87', 'request_id': 'req-1204d567-5c10-4706-a104-82ca92d08551'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:52.058590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] The token cache is being invalidate because project 2eefb400a71e45c4996f6894680f1d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:52.058940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:52.059650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-1204d567-5c10-4706-a104-82ca92d08551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:52.061206 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1320/2619] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:51 2022] DELETE /identity/v3/projects/2eefb400a71e45c4996f6894680f1d87 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:52.066258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7697f17-0675-4ee3-a368-2cfa1e1ec213 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.066752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7697f17-0675-4ee3-a368-2cfa1e1ec213 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.066959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7697f17-0675-4ee3-a368-2cfa1e1ec213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.087828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f7697f17-0675-4ee3-a368-2cfa1e1ec213 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:52.089036 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f7697f17-0675-4ee3-a368-2cfa1e1ec213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:52.149734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1300/2620] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:52.159843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8aeb1-9741-4896-8b6b-01d1f2149959 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.160650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8aeb1-9741-4896-8b6b-01d1f2149959 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.160650 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8aeb1-9741-4896-8b6b-01d1f2149959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.181651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c8b8aeb1-9741-4896-8b6b-01d1f2149959 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:52.182933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c8b8aeb1-9741-4896-8b6b-01d1f2149959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:52.191177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1321/2621] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:52.195612 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7697f17-0675-4ee3-a368-2cfa1e1ec213 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:52.206795 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7697f17-0675-4ee3-a368-2cfa1e1ec213 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:52.211771 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:52.212374 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.212652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.212953 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.213621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:52.213967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:52.216370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:52.229152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1301/2622] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:56:52.233461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8aeb1-9741-4896-8b6b-01d1f2149959 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:52.242756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8aeb1-9741-4896-8b6b-01d1f2149959 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:52.247540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fffc2-6e9f-4c43-928d-641e0ee4b40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:52.248135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fffc2-6e9f-4c43-928d-641e0ee4b40c admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.248416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fffc2-6e9f-4c43-928d-641e0ee4b40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.248692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fffc2-6e9f-4c43-928d-641e0ee4b40c admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.249328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fffc2-6e9f-4c43-928d-641e0ee4b40c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:52.251432 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fffc2-6e9f-4c43-928d-641e0ee4b40c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:52.338438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1322/2623] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:52 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:52.342766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:52.353011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b7ea-b273-4565-8df6-c6156ff2f032 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:52.388601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:52.415384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be120587-0c47-403d-bdd4-c671d303b2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:52.415981 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120587-0c47-403d-bdd4-c671d303b2ee admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.416257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120587-0c47-403d-bdd4-c671d303b2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.416528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120587-0c47-403d-bdd4-c671d303b2ee admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.417158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be120587-0c47-403d-bdd4-c671d303b2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:52.419127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be120587-0c47-403d-bdd4-c671d303b2ee admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:52.551792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1302/2624] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:52 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:52.556281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fffc2-6e9f-4c43-928d-641e0ee4b40c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:52.568179 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fffc2-6e9f-4c43-928d-641e0ee4b40c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:52.573111 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a418a3c-fa27-4f81-896e-baeac18b473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:52.573716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a418a3c-fa27-4f81-896e-baeac18b473a admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.574027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a418a3c-fa27-4f81-896e-baeac18b473a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.574301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a418a3c-fa27-4f81-896e-baeac18b473a admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.575522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a418a3c-fa27-4f81-896e-baeac18b473a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:52.577115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a418a3c-fa27-4f81-896e-baeac18b473a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:52.714075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1323/2625] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:52.718404 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be120587-0c47-403d-bdd4-c671d303b2ee admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:52.728927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be120587-0c47-403d-bdd4-c671d303b2ee admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:52.737523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866803e-0f94-4ce9-a7ae-2c5b0880d134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:52.737523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866803e-0f94-4ce9-a7ae-2c5b0880d134 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.737523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866803e-0f94-4ce9-a7ae-2c5b0880d134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.737523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866803e-0f94-4ce9-a7ae-2c5b0880d134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.737966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866803e-0f94-4ce9-a7ae-2c5b0880d134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:52.737966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866803e-0f94-4ce9-a7ae-2c5b0880d134 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:52.749396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1303/2626] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:52.753924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a418a3c-fa27-4f81-896e-baeac18b473a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:52.760299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:52.764372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a418a3c-fa27-4f81-896e-baeac18b473a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:52.769073 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ee990-58f6-42f1-b525-0ee23e1b8594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:52.769716 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ee990-58f6-42f1-b525-0ee23e1b8594 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.770028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ee990-58f6-42f1-b525-0ee23e1b8594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.770302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ee990-58f6-42f1-b525-0ee23e1b8594 admin admin] PATH_INFO: `/v3/projects/15f76c2378be47d383eff52c80b33e36/users/633698e054284d4e98bdcadc0a62d2ac/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.784943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ee990-58f6-42f1-b525-0ee23e1b8594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15f76c2378be47d383eff52c80b33e36, user_id=633698e054284d4e98bdcadc0a62d2ac, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:52.787615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ee990-58f6-42f1-b525-0ee23e1b8594 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:52.854316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1324/2627] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:52 2022] PUT /identity/v3/projects/15f76c2378be47d383eff52c80b33e36/users/633698e054284d4e98bdcadc0a62d2ac/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:52.865534 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129373f-7dc7-40df-b148-b6bfd5819115 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:52.865820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129373f-7dc7-40df-b148-b6bfd5819115 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:52.866130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129373f-7dc7-40df-b148-b6bfd5819115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:52.895577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f129373f-7dc7-40df-b148-b6bfd5819115 None None] MFA Rules not processed for user `633698e054284d4e98bdcadc0a62d2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:52.896793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f129373f-7dc7-40df-b148-b6bfd5819115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:52.963263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1304/2628] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:52.968194 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ee990-58f6-42f1-b525-0ee23e1b8594 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.040458 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ee990-58f6-42f1-b525-0ee23e1b8594 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.046464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.047098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.047379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.047655 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.048281 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.050263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.136092 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1325/2629] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:52 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:53.140174 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129373f-7dc7-40df-b148-b6bfd5819115 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.150464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.152287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:53.153518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129373f-7dc7-40df-b148-b6bfd5819115 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.212456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c9aeb-5b58-4671-adad-a752b0dcee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.213072 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c9aeb-5b58-4671-adad-a752b0dcee07 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.213356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c9aeb-5b58-4671-adad-a752b0dcee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.213632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c9aeb-5b58-4671-adad-a752b0dcee07 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.214406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c9aeb-5b58-4671-adad-a752b0dcee07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.217094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c9aeb-5b58-4671-adad-a752b0dcee07 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.220219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe454-7dae-4bab-94f8-769954f4c273 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.224831 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.225799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.225799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.226067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] PATH_INFO: `/v3/users/e0da0b71befd478b8be9063d8a339537` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.227160 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0da0b71befd478b8be9063d8a339537)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.229087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.326611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1305/2630] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:53 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:53.330609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c9aeb-5b58-4671-adad-a752b0dcee07 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.341611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c9aeb-5b58-4671-adad-a752b0dcee07 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.346402 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197316f2-732c-4448-bdd8-54ce80c74265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.347033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197316f2-732c-4448-bdd8-54ce80c74265 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.347310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197316f2-732c-4448-bdd8-54ce80c74265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.347579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197316f2-732c-4448-bdd8-54ce80c74265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.348211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197316f2-732c-4448-bdd8-54ce80c74265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.349782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197316f2-732c-4448-bdd8-54ce80c74265 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.361559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1306/2631] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:53.365332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197316f2-732c-4448-bdd8-54ce80c74265 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.375240 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197316f2-732c-4448-bdd8-54ce80c74265 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.379988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf969ce-68da-430a-8f6f-f385aff3f5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.380560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf969ce-68da-430a-8f6f-f385aff3f5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.380841 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf969ce-68da-430a-8f6f-f385aff3f5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.381115 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf969ce-68da-430a-8f6f-f385aff3f5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.381743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf969ce-68da-430a-8f6f-f385aff3f5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.383357 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf969ce-68da-430a-8f6f-f385aff3f5cd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.394950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1307/2632] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:53.398743 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf969ce-68da-430a-8f6f-f385aff3f5cd admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.410330 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf969ce-68da-430a-8f6f-f385aff3f5cd admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.410946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0da0b71befd478b8be9063d8a339537'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:53.416435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7b3e0-4a75-4daa-8257-f586242a9165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.417094 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b3e0-4a75-4daa-8257-f586242a9165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.417378 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b3e0-4a75-4daa-8257-f586242a9165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.417651 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b3e0-4a75-4daa-8257-f586242a9165 admin admin] PATH_INFO: `/v3/projects/a7d71d65425f44c89db5a56b97385160/users/40ba9704e7a846a5b9b2de684a5edc19/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.421902 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:53.431809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0da0b71befd478b8be9063d8a339537'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:53.434953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:53.435290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7b3e0-4a75-4daa-8257-f586242a9165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d71d65425f44c89db5a56b97385160, user_id=40ba9704e7a846a5b9b2de684a5edc19, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.438707 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7b3e0-4a75-4daa-8257-f586242a9165 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.505028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0da0b71befd478b8be9063d8a339537'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:53.505698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0da0b71befd478b8be9063d8a339537'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:53.506436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-62ebb6f4-1042-4b52-96de-a9ebd4bffce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0da0b71befd478b8be9063d8a339537'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:53.511621 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1308/2633] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:53 2022] PUT /identity/v3/projects/a7d71d65425f44c89db5a56b97385160/users/40ba9704e7a846a5b9b2de684a5edc19/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:53.522843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363128da-57fd-4e5a-9976-2bf807ce34a6 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.523125 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363128da-57fd-4e5a-9976-2bf807ce34a6 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.523396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363128da-57fd-4e5a-9976-2bf807ce34a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.523634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1326/2634] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:53 2022] DELETE /identity/v3/users/e0da0b71befd478b8be9063d8a339537 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:53.529098 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2130b-2bd9-4809-b872-5c19096d068e None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.529393 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2130b-2bd9-4809-b872-5c19096d068e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.529667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2130b-2bd9-4809-b872-5c19096d068e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.549017 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-abb2130b-2bd9-4809-b872-5c19096d068e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:53.550292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-abb2130b-2bd9-4809-b872-5c19096d068e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:53.552700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-363128da-57fd-4e5a-9976-2bf807ce34a6 None None] MFA Rules not processed for user `40ba9704e7a846a5b9b2de684a5edc19`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:53.553927 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-363128da-57fd-4e5a-9976-2bf807ce34a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:53.615516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1327/2635] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:53.625637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2130b-2bd9-4809-b872-5c19096d068e None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.627071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-abb2130b-2bd9-4809-b872-5c19096d068e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:53.629827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1309/2636] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:53.633516 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363128da-57fd-4e5a-9976-2bf807ce34a6 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.634895 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-363128da-57fd-4e5a-9976-2bf807ce34a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:53.649290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363128da-57fd-4e5a-9976-2bf807ce34a6 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.654854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b48ef-9cc7-40b0-badb-4f516fd250fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.655463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b48ef-9cc7-40b0-badb-4f516fd250fa admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.655742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b48ef-9cc7-40b0-badb-4f516fd250fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.656016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b48ef-9cc7-40b0-badb-4f516fd250fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.656641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b48ef-9cc7-40b0-badb-4f516fd250fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.658595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b48ef-9cc7-40b0-badb-4f516fd250fa admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.686766 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2130b-2bd9-4809-b872-5c19096d068e None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.690445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc352d-340e-4294-ae10-ae498c60c504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.691112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc352d-340e-4294-ae10-ae498c60c504 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.691337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc352d-340e-4294-ae10-ae498c60c504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.691610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc352d-340e-4294-ae10-ae498c60c504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.701525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc352d-340e-4294-ae10-ae498c60c504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.702975 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc352d-340e-4294-ae10-ae498c60c504 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.712629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1328/2637] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:53.730040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1310/2638] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:53 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:53.736118 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc352d-340e-4294-ae10-ae498c60c504 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.746465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc352d-340e-4294-ae10-ae498c60c504 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.811069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f2b54-5df1-47a5-bb4b-3ecfa43fa6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.811695 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f2b54-5df1-47a5-bb4b-3ecfa43fa6e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.811976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f2b54-5df1-47a5-bb4b-3ecfa43fa6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.812254 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f2b54-5df1-47a5-bb4b-3ecfa43fa6e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.812902 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f2b54-5df1-47a5-bb4b-3ecfa43fa6e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.814867 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f2b54-5df1-47a5-bb4b-3ecfa43fa6e9 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.926185 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1329/2639] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:53 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:53.930394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b48ef-9cc7-40b0-badb-4f516fd250fa admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.943389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b48ef-9cc7-40b0-badb-4f516fd250fa admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.953602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545b03f-af6d-45ff-bb15-923e84882c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.954469 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b03f-af6d-45ff-bb15-923e84882c8e admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.954905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b03f-af6d-45ff-bb15-923e84882c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.955230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b03f-af6d-45ff-bb15-923e84882c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:53.955905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545b03f-af6d-45ff-bb15-923e84882c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:53.957547 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545b03f-af6d-45ff-bb15-923e84882c8e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:53.968610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1311/2640] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:53.972824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f2b54-5df1-47a5-bb4b-3ecfa43fa6e9 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:53.983480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f2b54-5df1-47a5-bb4b-3ecfa43fa6e9 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:53.988981 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:53.989916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:53.990233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:53.990511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] PATH_INFO: `/v3/projects/f37c9e6aed544f75843279029cc3c975/users/ccb44aaac79a44319ad33ec1b39e3c0d/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:54.007009 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37c9e6aed544f75843279029cc3c975, user_id=ccb44aaac79a44319ad33ec1b39e3c0d, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:54.009710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:54.012936 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:54.134211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1330/2641] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:53 2022] PUT /identity/v3/projects/f37c9e6aed544f75843279029cc3c975/users/ccb44aaac79a44319ad33ec1b39e3c0d/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:54.139491 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545b03f-af6d-45ff-bb15-923e84882c8e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:54.149753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545b03f-af6d-45ff-bb15-923e84882c8e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:54.179124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:54.180997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:54.217442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de82ee-e064-4961-923e-26b3e6e251f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:54.218188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de82ee-e064-4961-923e-26b3e6e251f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:54.218462 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de82ee-e064-4961-923e-26b3e6e251f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:54.218726 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de82ee-e064-4961-923e-26b3e6e251f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:54.219388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de82ee-e064-4961-923e-26b3e6e251f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:54.221117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de82ee-e064-4961-923e-26b3e6e251f3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:54.232976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1312/2642] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:54.239087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de82ee-e064-4961-923e-26b3e6e251f3 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:54.250626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de82ee-e064-4961-923e-26b3e6e251f3 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:54.255313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ace637-16ad-4649-a2b2-c625847492e4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:54.255809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052adde-d5e5-474f-ae62-da20a16fb8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:54.256158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052adde-d5e5-474f-ae62-da20a16fb8cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:54.256440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052adde-d5e5-474f-ae62-da20a16fb8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:54.256719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052adde-d5e5-474f-ae62-da20a16fb8cb admin admin] PATH_INFO: `/v3/projects/f37c9e6aed544f75843279029cc3c975/users/ccb44aaac79a44319ad33ec1b39e3c0d/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:54.258862 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052adde-d5e5-474f-ae62-da20a16fb8cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37c9e6aed544f75843279029cc3c975, user_id=ccb44aaac79a44319ad33ec1b39e3c0d, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:54.260093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:54.260712 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:54.260988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:54.261261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] PATH_INFO: `/v3/projects/b2402696d3094e30a080535b7e9b4027` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:54.261475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052adde-d5e5-474f-ae62-da20a16fb8cb admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:54.262919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2402696d3094e30a080535b7e9b4027)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:54.265448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:54.272520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1313/2643] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:54 2022] PUT /identity/v3/projects/f37c9e6aed544f75843279029cc3c975/users/ccb44aaac79a44319ad33ec1b39e3c0d/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:54.283241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181a8cd-77b5-4cf7-82dc-5c9bcbab1834 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:54.283526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181a8cd-77b5-4cf7-82dc-5c9bcbab1834 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:54.283797 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181a8cd-77b5-4cf7-82dc-5c9bcbab1834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:54.313063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-7181a8cd-77b5-4cf7-82dc-5c9bcbab1834 None None] MFA Rules not processed for user `ccb44aaac79a44319ad33ec1b39e3c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:54.314327 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7181a8cd-77b5-4cf7-82dc-5c9bcbab1834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:54.328645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:54.384580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1314/2644] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:54.396274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181a8cd-77b5-4cf7-82dc-5c9bcbab1834 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:54.397616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-7181a8cd-77b5-4cf7-82dc-5c9bcbab1834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:54.421040 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2402696d3094e30a080535b7e9b4027', 'request_id': 'req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.456883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181a8cd-77b5-4cf7-82dc-5c9bcbab1834 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:54.476522 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2402696d3094e30a080535b7e9b4027', 'request_id': 'req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.484099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2402696d3094e30a080535b7e9b4027', 'request_id': 'req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.495210 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] The token cache is being invalidate because project b2402696d3094e30a080535b7e9b4027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:54.495508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.496997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.498956 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1331/2645] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:54 2022] DELETE /identity/v3/projects/b2402696d3094e30a080535b7e9b4027 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:54.502825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:54.504207 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:54.508623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:54.509242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:54.510292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:54.510292 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:54.512697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:54.570605 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b9f7e-0d8d-47e6-b6be-cf0cae9163aa admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:54.576099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:54.576639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:54.576758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:54.576993 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:54.577261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] PATH_INFO: `/v3/users/c81eae8834374ac68506cbc462c04946` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:54.578182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:54.578417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81eae8834374ac68506cbc462c04946)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:54.580305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:54.587752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1315/2646] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:54.718216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:54.818022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81eae8834374ac68506cbc462c04946'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.830115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81eae8834374ac68506cbc462c04946'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.973836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81eae8834374ac68506cbc462c04946'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.974479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81eae8834374ac68506cbc462c04946'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.975105 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-399f38d1-7517-418f-8d65-d211a66bc3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81eae8834374ac68506cbc462c04946'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:54.989072 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1332/2647] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:54 2022] DELETE /identity/v3/users/c81eae8834374ac68506cbc462c04946 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:55.456434 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:55.469708 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981b03d-b4ff-4fe8-bc98-9141e1cbf552 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:55.502313 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:55.536003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:55.536711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:55.537058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:55.537400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] PATH_INFO: `/v3/projects/626d25a7152a403586ef87a9d29d7e75` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:55.538682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=626d25a7152a403586ef87a9d29d7e75)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:55.540890 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:55.698197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626d25a7152a403586ef87a9d29d7e75', 'request_id': 'req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:55.768045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626d25a7152a403586ef87a9d29d7e75', 'request_id': 'req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:55.774854 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626d25a7152a403586ef87a9d29d7e75', 'request_id': 'req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:55.786100 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] The token cache is being invalidate because project 626d25a7152a403586ef87a9d29d7e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:55.786394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:55.787037 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8d57791a-afc4-417b-9e02-0a1dd73dc5f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:55.788627 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1316/2648] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:55 2022] DELETE /identity/v3/projects/626d25a7152a403586ef87a9d29d7e75 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:55.794288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b36bb-d5ae-4f2a-90b1-24f51de4e877 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:55.794804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b36bb-d5ae-4f2a-90b1-24f51de4e877 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:55.794907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b36bb-d5ae-4f2a-90b1-24f51de4e877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:55.821158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-bb5b36bb-d5ae-4f2a-90b1-24f51de4e877 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:55.821158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-bb5b36bb-d5ae-4f2a-90b1-24f51de4e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:55.879693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1333/2649] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:55.887550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc8411-663f-4565-bc86-3b9ed65e6888 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:55.887838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc8411-663f-4565-bc86-3b9ed65e6888 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:55.888117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc8411-663f-4565-bc86-3b9ed65e6888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:55.908384 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e4dc8411-663f-4565-bc86-3b9ed65e6888 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:55.909614 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e4dc8411-663f-4565-bc86-3b9ed65e6888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:55.924091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1317/2650] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:55.931843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b36bb-d5ae-4f2a-90b1-24f51de4e877 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:55.945223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b36bb-d5ae-4f2a-90b1-24f51de4e877 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:55.950472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:55.952116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:55.952116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:55.952116 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:55.952413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:55.952631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:55.954280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:55.967155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1334/2651] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:55.971742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc8411-663f-4565-bc86-3b9ed65e6888 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:55.981724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc8411-663f-4565-bc86-3b9ed65e6888 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:55.986595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90ed68-2e7c-4644-b32b-32f38de2e382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:55.987189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90ed68-2e7c-4644-b32b-32f38de2e382 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:55.987467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90ed68-2e7c-4644-b32b-32f38de2e382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:55.987768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90ed68-2e7c-4644-b32b-32f38de2e382 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:55.988391 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90ed68-2e7c-4644-b32b-32f38de2e382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:55.990317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90ed68-2e7c-4644-b32b-32f38de2e382 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:56.110820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1318/2652] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:55 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:56.115115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:56.125387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5035322-0d8a-4de6-90e9-31b1b7112270 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:56.181466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a2454-f2bf-42b7-81ab-733db6a366f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:56.182099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2454-f2bf-42b7-81ab-733db6a366f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:56.182383 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2454-f2bf-42b7-81ab-733db6a366f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:56.182660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2454-f2bf-42b7-81ab-733db6a366f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:56.183294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a2454-f2bf-42b7-81ab-733db6a366f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:56.186164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a2454-f2bf-42b7-81ab-733db6a366f8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:56.283736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1335/2653] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:56 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:56.288046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90ed68-2e7c-4644-b32b-32f38de2e382 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:56.300566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90ed68-2e7c-4644-b32b-32f38de2e382 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:56.305827 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b030ed-2648-4199-b83d-257039be939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:56.309035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b030ed-2648-4199-b83d-257039be939c admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:56.309035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b030ed-2648-4199-b83d-257039be939c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:56.309035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b030ed-2648-4199-b83d-257039be939c admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:56.309035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b030ed-2648-4199-b83d-257039be939c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:56.309920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b030ed-2648-4199-b83d-257039be939c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:56.324893 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1319/2654] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:56.326915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a2454-f2bf-42b7-81ab-733db6a366f8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:56.338090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a2454-f2bf-42b7-81ab-733db6a366f8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:56.357922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c001fd5-d4c5-47c9-a7e1-06addfe3ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:56.357922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001fd5-d4c5-47c9-a7e1-06addfe3ac6b admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:56.357922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001fd5-d4c5-47c9-a7e1-06addfe3ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:56.357922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001fd5-d4c5-47c9-a7e1-06addfe3ac6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:56.358562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c001fd5-d4c5-47c9-a7e1-06addfe3ac6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:56.358562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c001fd5-d4c5-47c9-a7e1-06addfe3ac6b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:56.363326 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1336/2655] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:56.364314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b030ed-2648-4199-b83d-257039be939c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:56.374853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b030ed-2648-4199-b83d-257039be939c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:56.379481 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:56.380147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:56.380412 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:56.380688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] PATH_INFO: `/v3/projects/779afdc10a85445e99b8356bc35f2ef6/users/6333a190af424f11801f2a3544f1625b/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:56.396389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=779afdc10a85445e99b8356bc35f2ef6, user_id=6333a190af424f11801f2a3544f1625b, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:56.399181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:56.515802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1320/2656] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:56 2022] PUT /identity/v3/projects/779afdc10a85445e99b8356bc35f2ef6/users/6333a190af424f11801f2a3544f1625b/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:56.528097 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ec212-b49f-4b43-b052-d6d8765710ab None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:56.528416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ec212-b49f-4b43-b052-d6d8765710ab None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:56.528769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ec212-b49f-4b43-b052-d6d8765710ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:56.549988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:56.563071 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-de2ec212-b49f-4b43-b052-d6d8765710ab None None] MFA Rules not processed for user `6333a190af424f11801f2a3544f1625b`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:56.564386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-de2ec212-b49f-4b43-b052-d6d8765710ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:56.635658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:56.637900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:56.638690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1337/2657] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:56.643364 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ec212-b49f-4b43-b052-d6d8765710ab None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:56.695113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:56.701151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b3148-8d31-4518-90c0-8d23f5df8915 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:56.704856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:56.705454 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:56.705738 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:56.706034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:56.706832 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ec212-b49f-4b43-b052-d6d8765710ab None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:56.707786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:56.711973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6106cad-01cb-4473-aa78-3089f98c8ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:56.712574 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6106cad-01cb-4473-aa78-3089f98c8ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:56.712854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6106cad-01cb-4473-aa78-3089f98c8ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:56.713131 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6106cad-01cb-4473-aa78-3089f98c8ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:56.713749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6106cad-01cb-4473-aa78-3089f98c8ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:56.716286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6106cad-01cb-4473-aa78-3089f98c8ab5 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:56.769321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:56.770796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:56.780518 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1321/2658] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:56.804982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1338/2659] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:56 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:56.809478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:56.820683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca05aa-3115-44d1-940a-bacb48519204 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:56.879098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee8ba8-a788-441d-82f1-a8644ec5c3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:56.879701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee8ba8-a788-441d-82f1-a8644ec5c3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:56.879980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee8ba8-a788-441d-82f1-a8644ec5c3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:56.880250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee8ba8-a788-441d-82f1-a8644ec5c3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:56.880873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee8ba8-a788-441d-82f1-a8644ec5c3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:56.882859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee8ba8-a788-441d-82f1-a8644ec5c3c1 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:56.991439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1322/2660] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:56 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:56.992584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6106cad-01cb-4473-aa78-3089f98c8ab5 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:57.003317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6106cad-01cb-4473-aa78-3089f98c8ab5 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:57.008039 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0550e-fd6b-453b-b39d-5fdfc602262f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:57.008642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0550e-fd6b-453b-b39d-5fdfc602262f admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:57.008916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0550e-fd6b-453b-b39d-5fdfc602262f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:57.009188 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0550e-fd6b-453b-b39d-5fdfc602262f admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:57.009840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0550e-fd6b-453b-b39d-5fdfc602262f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:57.011447 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0550e-fd6b-453b-b39d-5fdfc602262f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:57.021916 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1339/2661] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:57.026049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee8ba8-a788-441d-82f1-a8644ec5c3c1 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:57.037201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee8ba8-a788-441d-82f1-a8644ec5c3c1 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:57.043880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06c972-afc0-4edd-a885-69e0aeb866b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:57.043880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06c972-afc0-4edd-a885-69e0aeb866b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:57.044158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06c972-afc0-4edd-a885-69e0aeb866b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:57.044270 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06c972-afc0-4edd-a885-69e0aeb866b3 admin admin] PATH_INFO: `/v3/projects/920f3da073f2481f95d11af8ad7bb390/users/8148509a79e8420692d8d139e10a6cf9/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:57.060166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06c972-afc0-4edd-a885-69e0aeb866b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=920f3da073f2481f95d11af8ad7bb390, user_id=8148509a79e8420692d8d139e10a6cf9, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:57.062848 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06c972-afc0-4edd-a885-69e0aeb866b3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:57.148695 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1323/2662] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:57 2022] PUT /identity/v3/projects/920f3da073f2481f95d11af8ad7bb390/users/8148509a79e8420692d8d139e10a6cf9/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:57.153625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0550e-fd6b-453b-b39d-5fdfc602262f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:57.163989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0550e-fd6b-453b-b39d-5fdfc602262f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:57.223972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7552ae9-c3c7-40d7-813c-39bffdab36bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:57.224630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7552ae9-c3c7-40d7-813c-39bffdab36bf admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:57.224881 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7552ae9-c3c7-40d7-813c-39bffdab36bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:57.225153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7552ae9-c3c7-40d7-813c-39bffdab36bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:57.225793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7552ae9-c3c7-40d7-813c-39bffdab36bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:57.227521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7552ae9-c3c7-40d7-813c-39bffdab36bf admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:57.237818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1340/2663] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:57.242236 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06c972-afc0-4edd-a885-69e0aeb866b3 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:57.252913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06c972-afc0-4edd-a885-69e0aeb866b3 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:57.257520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:57.258180 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:57.258458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:57.258724 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] PATH_INFO: `/v3/projects/920f3da073f2481f95d11af8ad7bb390/users/8148509a79e8420692d8d139e10a6cf9/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:57.260659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=920f3da073f2481f95d11af8ad7bb390, user_id=8148509a79e8420692d8d139e10a6cf9, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:57.264075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:57.274736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1324/2664] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:57 2022] PUT /identity/v3/projects/920f3da073f2481f95d11af8ad7bb390/users/8148509a79e8420692d8d139e10a6cf9/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:57.285510 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d00c99-bd03-4fd0-8742-6bbe236276d9 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:57.285923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d00c99-bd03-4fd0-8742-6bbe236276d9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:57.286273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d00c99-bd03-4fd0-8742-6bbe236276d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:57.307241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:57.316020 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-02d00c99-bd03-4fd0-8742-6bbe236276d9 None None] MFA Rules not processed for user `8148509a79e8420692d8d139e10a6cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:57.317233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-02d00c99-bd03-4fd0-8742-6bbe236276d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:57.386056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1341/2665] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:56:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:57.397922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:57.399623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:57.460049 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff98eaa-2a66-48fc-a983-1033f700a721 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:57.463791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfc034-dbc8-4940-8e00-25755b4422d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:57.464423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfc034-dbc8-4940-8e00-25755b4422d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:57.464701 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfc034-dbc8-4940-8e00-25755b4422d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:57.465659 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfc034-dbc8-4940-8e00-25755b4422d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:57.471091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:57.522593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfc034-dbc8-4940-8e00-25755b4422d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:57.523712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfc034-dbc8-4940-8e00-25755b4422d9 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:57.534608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1325/2666] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:58.617451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d00c99-bd03-4fd0-8742-6bbe236276d9 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:58.628273 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d00c99-bd03-4fd0-8742-6bbe236276d9 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:58.631763 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:58.632359 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:58.632630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:58.632898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:58.634651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:58.699162 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:58.701291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:58.710860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1342/2667] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:02.820431 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfc034-dbc8-4940-8e00-25755b4422d9 service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:02.832422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfc034-dbc8-4940-8e00-25755b4422d9 service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:02.833476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5d1c-c03e-4377-8c2b-164c0ff18c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:02.834362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5d1c-c03e-4377-8c2b-164c0ff18c6c service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:02.834362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5d1c-c03e-4377-8c2b-164c0ff18c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:02.834554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5d1c-c03e-4377-8c2b-164c0ff18c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:02.844610 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb5d1c-c03e-4377-8c2b-164c0ff18c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:02.846083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb5d1c-c03e-4377-8c2b-164c0ff18c6c service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:02.856083 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1326/2668] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:03.320840 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:03.322342 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:03.388353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8b246-bf9d-4468-b99f-c39642c79677 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:03.393423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:03.394079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:03.394355 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:03.394623 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] PATH_INFO: `/v3/users/633698e054284d4e98bdcadc0a62d2ac` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:03.395734 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=633698e054284d4e98bdcadc0a62d2ac)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:03.397646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:03.769163 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:03.783165 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '633698e054284d4e98bdcadc0a62d2ac'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:03.795800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '633698e054284d4e98bdcadc0a62d2ac'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:03.875814 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '633698e054284d4e98bdcadc0a62d2ac'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:03.876422 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '633698e054284d4e98bdcadc0a62d2ac'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:03.877169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '633698e054284d4e98bdcadc0a62d2ac'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:03.891942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1343/2669] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:03 2022] DELETE /identity/v3/users/633698e054284d4e98bdcadc0a62d2ac => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:03.897062 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632036d-9840-4c19-9391-6237f6611211 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:03.897604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632036d-9840-4c19-9391-6237f6611211 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:03.897800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632036d-9840-4c19-9391-6237f6611211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:03.920495 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a632036d-9840-4c19-9391-6237f6611211 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:03.921898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a632036d-9840-4c19-9391-6237f6611211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:03.986613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1327/2670] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:03.997270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:03.998880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:04.062828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251e2d-88f2-4a7e-a760-e383d7f4c719 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:04.066608 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9e000-48ca-4d1a-a787-1bb29fa2f7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:04.067313 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9e000-48ca-4d1a-a787-1bb29fa2f7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:04.067586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9e000-48ca-4d1a-a787-1bb29fa2f7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:04.067854 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9e000-48ca-4d1a-a787-1bb29fa2f7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:04.078126 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9e000-48ca-4d1a-a787-1bb29fa2f7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:04.079576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9e000-48ca-4d1a-a787-1bb29fa2f7d1 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:04.089354 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1344/2671] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:04.537984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632036d-9840-4c19-9391-6237f6611211 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:04.539526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a632036d-9840-4c19-9391-6237f6611211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:04.553425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632036d-9840-4c19-9391-6237f6611211 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:04.558552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:04.559354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:04.559609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:04.559894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] PATH_INFO: `/v3/projects/15f76c2378be47d383eff52c80b33e36` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:04.561064 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=15f76c2378be47d383eff52c80b33e36)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:04.564669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:04.689918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15f76c2378be47d383eff52c80b33e36', 'request_id': 'req-400eafc0-ba77-435f-93b4-130cedd3b66d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:04.770379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15f76c2378be47d383eff52c80b33e36', 'request_id': 'req-400eafc0-ba77-435f-93b4-130cedd3b66d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:04.776652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15f76c2378be47d383eff52c80b33e36', 'request_id': 'req-400eafc0-ba77-435f-93b4-130cedd3b66d'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:04.789046 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] The token cache is being invalidate because project 15f76c2378be47d383eff52c80b33e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:04.789376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:04.790054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:04.791573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1328/2672] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:04 2022] DELETE /identity/v3/projects/15f76c2378be47d383eff52c80b33e36 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:04.795673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9e000-48ca-4d1a-a787-1bb29fa2f7d1 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:04.806300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9e000-48ca-4d1a-a787-1bb29fa2f7d1 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:04.865257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:04.865946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:04.866216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:04.866489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] PATH_INFO: `/v3/users/40ba9704e7a846a5b9b2de684a5edc19` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:04.867629 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ba9704e7a846a5b9b2de684a5edc19)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:04.869643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:05.120721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ba9704e7a846a5b9b2de684a5edc19'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.125361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:05.134025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ba9704e7a846a5b9b2de684a5edc19'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.191018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ba9704e7a846a5b9b2de684a5edc19'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.192325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ba9704e7a846a5b9b2de684a5edc19'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.192942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ba9704e7a846a5b9b2de684a5edc19'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.207149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1345/2673] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:04 2022] DELETE /identity/v3/users/40ba9704e7a846a5b9b2de684a5edc19 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:05.648172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:05.649921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:05.680257 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:05.743216 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eafc0-ba77-435f-93b4-130cedd3b66d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:05.800050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:05.800680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:05.800964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:05.801241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] PATH_INFO: `/v3/projects/a7d71d65425f44c89db5a56b97385160` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:05.802424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7d71d65425f44c89db5a56b97385160)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:05.804359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:05.910630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7d71d65425f44c89db5a56b97385160', 'request_id': 'req-bfe337ee-6896-4306-a891-a13aba5008f8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.967704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7d71d65425f44c89db5a56b97385160', 'request_id': 'req-bfe337ee-6896-4306-a891-a13aba5008f8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.974370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7d71d65425f44c89db5a56b97385160', 'request_id': 'req-bfe337ee-6896-4306-a891-a13aba5008f8'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.986444 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] The token cache is being invalidate because project a7d71d65425f44c89db5a56b97385160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:05.986739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.987379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:05.988860 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1329/2674] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:05 2022] DELETE /identity/v3/projects/a7d71d65425f44c89db5a56b97385160 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:05.992873 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:05.994371 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:06.051186 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c2cea-4255-41a7-8dff-334a6815efc3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:06.107730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:06.108368 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:06.109284 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:06.109559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] PATH_INFO: `/v3/users/ccb44aaac79a44319ad33ec1b39e3c0d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:06.110890 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb44aaac79a44319ad33ec1b39e3c0d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:06.112804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:06.310064 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb44aaac79a44319ad33ec1b39e3c0d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:06.323385 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb44aaac79a44319ad33ec1b39e3c0d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:06.402456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb44aaac79a44319ad33ec1b39e3c0d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:06.403080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb44aaac79a44319ad33ec1b39e3c0d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:06.403692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-410a8f30-5f42-4f6f-b7d6-a2c5fa0d62a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb44aaac79a44319ad33ec1b39e3c0d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:06.417771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1346/2675] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:05 2022] DELETE /identity/v3/users/ccb44aaac79a44319ad33ec1b39e3c0d => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:06.881741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:06.883461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:06.898698 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe337ee-6896-4306-a891-a13aba5008f8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:06.947018 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:06.961089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:06.961821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:06.962138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:06.963661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] PATH_INFO: `/v3/projects/f37c9e6aed544f75843279029cc3c975` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:06.963661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f37c9e6aed544f75843279029cc3c975)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:06.965500 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:07.142051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f37c9e6aed544f75843279029cc3c975', 'request_id': 'req-8381a793-65c2-4493-990f-bc86f8f4d1dd'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:07.233525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f37c9e6aed544f75843279029cc3c975', 'request_id': 'req-8381a793-65c2-4493-990f-bc86f8f4d1dd'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:07.240990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f37c9e6aed544f75843279029cc3c975', 'request_id': 'req-8381a793-65c2-4493-990f-bc86f8f4d1dd'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:07.253638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] The token cache is being invalidate because project f37c9e6aed544f75843279029cc3c975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:07.253967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:07.254885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8381a793-65c2-4493-990f-bc86f8f4d1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:07.256512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1330/2676] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:06 2022] DELETE /identity/v3/projects/f37c9e6aed544f75843279029cc3c975 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:07.262069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc205e-95ee-4179-b0f6-01ad63dd14a4 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:07.263302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc205e-95ee-4179-b0f6-01ad63dd14a4 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:07.263302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc205e-95ee-4179-b0f6-01ad63dd14a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:07.273310 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:07.327085 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-c5dc205e-95ee-4179-b0f6-01ad63dd14a4 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:07.328362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c5dc205e-95ee-4179-b0f6-01ad63dd14a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:07.389619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1347/2677] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:07.398150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21919-bdd2-498d-ab52-0f185c9f1513 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:07.398418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21919-bdd2-498d-ab52-0f185c9f1513 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:07.398698 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21919-bdd2-498d-ab52-0f185c9f1513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:07.419220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-3fa21919-bdd2-498d-ab52-0f185c9f1513 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:07.420448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3fa21919-bdd2-498d-ab52-0f185c9f1513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:07.429463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1331/2678] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:07.433807 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc205e-95ee-4179-b0f6-01ad63dd14a4 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:07.444399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc205e-95ee-4179-b0f6-01ad63dd14a4 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:07.448935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:07.449572 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:07.449877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:07.450170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:07.451192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:57:07.451518 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:07.453083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:07.468417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1348/2679] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:57:07.472194 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa21919-bdd2-498d-ab52-0f185c9f1513 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:07.479159 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa21919-bdd2-498d-ab52-0f185c9f1513 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:07.484423 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0eb6c-bd03-4734-a5ad-78c367b0e3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:07.485276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0eb6c-bd03-4734-a5ad-78c367b0e3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:07.485559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0eb6c-bd03-4734-a5ad-78c367b0e3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:07.486419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0eb6c-bd03-4734-a5ad-78c367b0e3b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:07.487057 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0eb6c-bd03-4734-a5ad-78c367b0e3b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:07.489105 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0eb6c-bd03-4734-a5ad-78c367b0e3b2 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:07.607514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1332/2680] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:57:07 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:57:07.612000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:07.622396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c289788-e02a-49d2-aab5-e5c95ffa962b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:07.683945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d8ac2-d61e-4d98-9a01-4d4ffe2a4afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:07.684547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d8ac2-d61e-4d98-9a01-4d4ffe2a4afd admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:07.684829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d8ac2-d61e-4d98-9a01-4d4ffe2a4afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:07.685100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d8ac2-d61e-4d98-9a01-4d4ffe2a4afd admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:07.685720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d8ac2-d61e-4d98-9a01-4d4ffe2a4afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:07.688077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d8ac2-d61e-4d98-9a01-4d4ffe2a4afd admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:07.874225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1349/2681] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:57:07 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:07.887877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0eb6c-bd03-4734-a5ad-78c367b0e3b2 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:07.899116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0eb6c-bd03-4734-a5ad-78c367b0e3b2 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:07.908063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550ddf5-989f-4930-b88e-ee01dfd2ff36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:07.908063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550ddf5-989f-4930-b88e-ee01dfd2ff36 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:07.908063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550ddf5-989f-4930-b88e-ee01dfd2ff36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:07.908063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550ddf5-989f-4930-b88e-ee01dfd2ff36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:07.908396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550ddf5-989f-4930-b88e-ee01dfd2ff36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:07.909218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550ddf5-989f-4930-b88e-ee01dfd2ff36 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:07.924082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1333/2682] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:57:07.928100 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d8ac2-d61e-4d98-9a01-4d4ffe2a4afd admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:07.938961 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d8ac2-d61e-4d98-9a01-4d4ffe2a4afd admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:07.944065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd414ae-939d-4965-8fee-b9e4ec068553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:07.944672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd414ae-939d-4965-8fee-b9e4ec068553 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:07.944951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd414ae-939d-4965-8fee-b9e4ec068553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:07.945270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd414ae-939d-4965-8fee-b9e4ec068553 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:07.945911 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd414ae-939d-4965-8fee-b9e4ec068553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:07.947490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd414ae-939d-4965-8fee-b9e4ec068553 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:07.960966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1350/2683] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:07.966448 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550ddf5-989f-4930-b88e-ee01dfd2ff36 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:07.977770 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550ddf5-989f-4930-b88e-ee01dfd2ff36 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:07.982409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c39c3-168c-4a26-a7b7-727968129299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:07.983063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c39c3-168c-4a26-a7b7-727968129299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:07.983472 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c39c3-168c-4a26-a7b7-727968129299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:07.983714 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c39c3-168c-4a26-a7b7-727968129299 admin admin] PATH_INFO: `/v3/projects/a7e1eeabfb73438998aa1b8ffbbef75a/users/d5b709ce4f04423c8d589a6b9a833179/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:07.999942 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c39c3-168c-4a26-a7b7-727968129299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e1eeabfb73438998aa1b8ffbbef75a, user_id=d5b709ce4f04423c8d589a6b9a833179, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:08.002740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c39c3-168c-4a26-a7b7-727968129299 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:08.091387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1334/2684] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:07 2022] PUT /identity/v3/projects/a7e1eeabfb73438998aa1b8ffbbef75a/users/d5b709ce4f04423c8d589a6b9a833179/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:08.103622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e171b-644e-48e2-b92a-bb2c9547bf21 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:08.103906 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e171b-644e-48e2-b92a-bb2c9547bf21 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:08.104182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e171b-644e-48e2-b92a-bb2c9547bf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:08.132500 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-f34e171b-644e-48e2-b92a-bb2c9547bf21 None None] MFA Rules not processed for user `d5b709ce4f04423c8d589a6b9a833179`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:08.133719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-f34e171b-644e-48e2-b92a-bb2c9547bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:08.167195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:08.201365 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1351/2685] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:08.206826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c39c3-168c-4a26-a7b7-727968129299 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:08.270853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c39c3-168c-4a26-a7b7-727968129299 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:08.275739 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdbdce-ce6f-4c81-8679-c108c8dc91f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:08.276390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdbdce-ce6f-4c81-8679-c108c8dc91f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:08.276678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdbdce-ce6f-4c81-8679-c108c8dc91f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:08.276952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdbdce-ce6f-4c81-8679-c108c8dc91f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:08.277586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdbdce-ce6f-4c81-8679-c108c8dc91f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:08.279579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdbdce-ce6f-4c81-8679-c108c8dc91f1 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:08.359772 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:08.367091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1335/2686] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:57:08 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:08.371437 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e171b-644e-48e2-b92a-bb2c9547bf21 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:08.383301 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e171b-644e-48e2-b92a-bb2c9547bf21 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:08.442399 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d4ec0-3a11-46cf-bb0e-b85065948564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:08.442928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d4ec0-3a11-46cf-bb0e-b85065948564 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:08.443212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d4ec0-3a11-46cf-bb0e-b85065948564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:08.443490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d4ec0-3a11-46cf-bb0e-b85065948564 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:08.444128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d4ec0-3a11-46cf-bb0e-b85065948564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:08.446086 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d4ec0-3a11-46cf-bb0e-b85065948564 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:08.556413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1352/2687] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:57:08 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:08.560468 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdbdce-ce6f-4c81-8679-c108c8dc91f1 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:08.572244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdbdce-ce6f-4c81-8679-c108c8dc91f1 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:08.577009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1154d-1fa9-422f-97b7-6a5d87a3b313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:08.577757 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1154d-1fa9-422f-97b7-6a5d87a3b313 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:08.578107 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1154d-1fa9-422f-97b7-6a5d87a3b313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:08.578349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1154d-1fa9-422f-97b7-6a5d87a3b313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:08.579188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1154d-1fa9-422f-97b7-6a5d87a3b313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:08.580618 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1154d-1fa9-422f-97b7-6a5d87a3b313 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:08.591628 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1336/2688] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:08.596502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d4ec0-3a11-46cf-bb0e-b85065948564 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:08.606974 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d4ec0-3a11-46cf-bb0e-b85065948564 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:08.612582 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9e817-fd4c-4dfb-b8c2-e53cc6262dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:08.613186 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e817-fd4c-4dfb-b8c2-e53cc6262dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:08.613470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e817-fd4c-4dfb-b8c2-e53cc6262dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:08.613748 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e817-fd4c-4dfb-b8c2-e53cc6262dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:08.614436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9e817-fd4c-4dfb-b8c2-e53cc6262dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:08.616025 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9e817-fd4c-4dfb-b8c2-e53cc6262dd6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:08.626897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1353/2689] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:08.631068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1154d-1fa9-422f-97b7-6a5d87a3b313 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:08.641606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1154d-1fa9-422f-97b7-6a5d87a3b313 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:08.646277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809e5df-9104-41a5-b1a5-84944b80982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:08.646928 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809e5df-9104-41a5-b1a5-84944b80982c admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:08.647206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809e5df-9104-41a5-b1a5-84944b80982c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:08.647478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809e5df-9104-41a5-b1a5-84944b80982c admin admin] PATH_INFO: `/v3/projects/84f268654e67490da489a09e32ff6da8/users/34aa5d3c3ac7414fb8a800757f40bbc9/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:08.662736 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809e5df-9104-41a5-b1a5-84944b80982c admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f268654e67490da489a09e32ff6da8, user_id=34aa5d3c3ac7414fb8a800757f40bbc9, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:08.665433 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809e5df-9104-41a5-b1a5-84944b80982c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:08.753502 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1337/2690] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:08 2022] PUT /identity/v3/projects/84f268654e67490da489a09e32ff6da8/users/34aa5d3c3ac7414fb8a800757f40bbc9/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:08.763971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc3f35-2854-4a56-bf9c-fa226ccb541d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:08.764262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc3f35-2854-4a56-bf9c-fa226ccb541d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:08.764554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc3f35-2854-4a56-bf9c-fa226ccb541d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:08.794627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-9abc3f35-2854-4a56-bf9c-fa226ccb541d None None] MFA Rules not processed for user `34aa5d3c3ac7414fb8a800757f40bbc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:08.795852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9abc3f35-2854-4a56-bf9c-fa226ccb541d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:08.860084 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1354/2691] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:08.864645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809e5df-9104-41a5-b1a5-84944b80982c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:08.931233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809e5df-9104-41a5-b1a5-84944b80982c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:08.936302 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb415a-a3af-4c6b-afc1-b5a8f8c4a53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:08.937089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb415a-a3af-4c6b-afc1-b5a8f8c4a53f admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:08.937380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb415a-a3af-4c6b-afc1-b5a8f8c4a53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:08.937649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb415a-a3af-4c6b-afc1-b5a8f8c4a53f admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:08.938323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb415a-a3af-4c6b-afc1-b5a8f8c4a53f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:08.940273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb415a-a3af-4c6b-afc1-b5a8f8c4a53f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:09.041460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1338/2692] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:57:08 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:09.042929 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc3f35-2854-4a56-bf9c-fa226ccb541d None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:09.055223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc3f35-2854-4a56-bf9c-fa226ccb541d None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:09.098021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:09.116260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce056c-70f4-495f-af06-4a0f56cd0364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:09.116865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce056c-70f4-495f-af06-4a0f56cd0364 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:09.117141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce056c-70f4-495f-af06-4a0f56cd0364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:09.117411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce056c-70f4-495f-af06-4a0f56cd0364 admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:09.118061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce056c-70f4-495f-af06-4a0f56cd0364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:09.119948 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce056c-70f4-495f-af06-4a0f56cd0364 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:09.234501 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1355/2693] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:57:09 2022] POST /identity/v3/users => generated 298 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:09.238585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb415a-a3af-4c6b-afc1-b5a8f8c4a53f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:09.249021 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb415a-a3af-4c6b-afc1-b5a8f8c4a53f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:09.254429 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345521f-2c16-40b2-98b5-299ee9ea98a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:09.255033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345521f-2c16-40b2-98b5-299ee9ea98a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:09.255307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345521f-2c16-40b2-98b5-299ee9ea98a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:09.255578 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345521f-2c16-40b2-98b5-299ee9ea98a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:09.256230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345521f-2c16-40b2-98b5-299ee9ea98a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:09.257822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345521f-2c16-40b2-98b5-299ee9ea98a7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:09.269290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1339/2694] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:09.273396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce056c-70f4-495f-af06-4a0f56cd0364 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:09.283719 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce056c-70f4-495f-af06-4a0f56cd0364 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:09.289088 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b28ab-ab30-44de-91c6-b1fbf1ea3015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:09.289758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b28ab-ab30-44de-91c6-b1fbf1ea3015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:09.290075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b28ab-ab30-44de-91c6-b1fbf1ea3015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:09.290351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b28ab-ab30-44de-91c6-b1fbf1ea3015 admin admin] PATH_INFO: `/v3/projects/b1ea4fb1269d40a2a4931ef1c85269fa/users/ad0682650b0142df97b138deeb64da02/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:09.305824 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b28ab-ab30-44de-91c6-b1fbf1ea3015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ea4fb1269d40a2a4931ef1c85269fa, user_id=ad0682650b0142df97b138deeb64da02, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:09.308636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b28ab-ab30-44de-91c6-b1fbf1ea3015 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:09.392640 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1356/2695] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:09 2022] PUT /identity/v3/projects/b1ea4fb1269d40a2a4931ef1c85269fa/users/ad0682650b0142df97b138deeb64da02/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:09.396720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345521f-2c16-40b2-98b5-299ee9ea98a7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:09.407050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345521f-2c16-40b2-98b5-299ee9ea98a7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:09.467122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6307e75-f616-4b40-acda-7a1e8ff4529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:09.467720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307e75-f616-4b40-acda-7a1e8ff4529c admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:09.467997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307e75-f616-4b40-acda-7a1e8ff4529c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:09.468410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307e75-f616-4b40-acda-7a1e8ff4529c admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:09.469419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6307e75-f616-4b40-acda-7a1e8ff4529c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:09.471108 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6307e75-f616-4b40-acda-7a1e8ff4529c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:09.480993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1340/2696] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:57:09.486074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b28ab-ab30-44de-91c6-b1fbf1ea3015 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:09.496795 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b28ab-ab30-44de-91c6-b1fbf1ea3015 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:09.502529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:09.503168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:09.503468 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:09.503729 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] PATH_INFO: `/v3/projects/b1ea4fb1269d40a2a4931ef1c85269fa/users/ad0682650b0142df97b138deeb64da02/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:09.505644 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ea4fb1269d40a2a4931ef1c85269fa, user_id=ad0682650b0142df97b138deeb64da02, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:09.508345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:09.519600 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1357/2697] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:09 2022] PUT /identity/v3/projects/b1ea4fb1269d40a2a4931ef1c85269fa/users/ad0682650b0142df97b138deeb64da02/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:09.530003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cdb7-5496-4cb1-8d31-d92322d3f3b6 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:09.530293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cdb7-5496-4cb1-8d31-d92322d3f3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:09.530599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cdb7-5496-4cb1-8d31-d92322d3f3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:09.559097 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-71f6cdb7-5496-4cb1-8d31-d92322d3f3b6 None None] MFA Rules not processed for user `ad0682650b0142df97b138deeb64da02`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:09.560320 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-71f6cdb7-5496-4cb1-8d31-d92322d3f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:09.582129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:09.632082 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1341/2698] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:09.646340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:09.648586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:09.724659 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b099e-c4bf-49e2-8584-43954b6f331d admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:09.728341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07e10f-c622-449b-9536-418ae19036bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:09.728942 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e10f-c622-449b-9536-418ae19036bd service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:09.729221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e10f-c622-449b-9536-418ae19036bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:09.729489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e10f-c622-449b-9536-418ae19036bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:09.787127 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07e10f-c622-449b-9536-418ae19036bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:09.788453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07e10f-c622-449b-9536-418ae19036bd service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:09.798125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1358/2699] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:10.206163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6cdb7-5496-4cb1-8d31-d92322d3f3b6 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:10.207653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-71f6cdb7-5496-4cb1-8d31-d92322d3f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:10.278263 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6cdb7-5496-4cb1-8d31-d92322d3f3b6 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:10.282974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:10.283602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:10.283905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:10.284149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] PATH_INFO: `/v3/users/6333a190af424f11801f2a3544f1625b` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:10.285331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6333a190af424f11801f2a3544f1625b)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:10.287311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:10.468666 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6333a190af424f11801f2a3544f1625b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:10.546106 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6333a190af424f11801f2a3544f1625b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:10.548291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6333a190af424f11801f2a3544f1625b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:10.559035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6333a190af424f11801f2a3544f1625b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:10.559558 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6333a190af424f11801f2a3544f1625b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:10.582395 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1342/2700] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:10 2022] DELETE /identity/v3/users/6333a190af424f11801f2a3544f1625b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:10.586671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f22c-76ec-44f7-a91f-152401906f73 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:10.587192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f22c-76ec-44f7-a91f-152401906f73 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:10.587410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f22c-76ec-44f7-a91f-152401906f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:10.607236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-14a1f22c-76ec-44f7-a91f-152401906f73 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:10.608488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-14a1f22c-76ec-44f7-a91f-152401906f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:10.634925 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:10.668782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1359/2701] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:10.679606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:10.681011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:10.726734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:10.743509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a68bf-e195-4a8f-b40f-9f7953e18cfe admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:10.747167 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d088a-6b94-46a4-8d43-42cc78a04b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:10.747794 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d088a-6b94-46a4-8d43-42cc78a04b09 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:10.748259 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d088a-6b94-46a4-8d43-42cc78a04b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:10.748383 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d088a-6b94-46a4-8d43-42cc78a04b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:10.758438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d088a-6b94-46a4-8d43-42cc78a04b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:10.759918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d088a-6b94-46a4-8d43-42cc78a04b09 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:10.773616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1343/2702] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:11.252379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1f22c-76ec-44f7-a91f-152401906f73 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:11.255492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-14a1f22c-76ec-44f7-a91f-152401906f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:11.266211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1f22c-76ec-44f7-a91f-152401906f73 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:11.272031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:11.272661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:11.272943 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:11.273215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] PATH_INFO: `/v3/projects/779afdc10a85445e99b8356bc35f2ef6` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:11.274389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=779afdc10a85445e99b8356bc35f2ef6)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:11.276344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:11.435038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '779afdc10a85445e99b8356bc35f2ef6', 'request_id': 'req-83cc5514-2bfb-4310-8879-45b1e4ba6d14'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.501940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '779afdc10a85445e99b8356bc35f2ef6', 'request_id': 'req-83cc5514-2bfb-4310-8879-45b1e4ba6d14'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.511714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '779afdc10a85445e99b8356bc35f2ef6', 'request_id': 'req-83cc5514-2bfb-4310-8879-45b1e4ba6d14'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.520229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] The token cache is being invalidate because project 779afdc10a85445e99b8356bc35f2ef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:11.520523 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.521181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.522693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1360/2703] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:11 2022] DELETE /identity/v3/projects/779afdc10a85445e99b8356bc35f2ef6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:11.526730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d088a-6b94-46a4-8d43-42cc78a04b09 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:11.537885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d088a-6b94-46a4-8d43-42cc78a04b09 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:11.602028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:11.602674 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:11.602933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:11.603202 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] PATH_INFO: `/v3/users/8148509a79e8420692d8d139e10a6cf9` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:11.604344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8148509a79e8420692d8d139e10a6cf9)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:11.606277 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:11.867797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:11.868316 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8148509a79e8420692d8d139e10a6cf9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.869530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:11.930755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc5514-2bfb-4310-8879-45b1e4ba6d14 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:11.934366 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:11.934967 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:11.935247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:11.935520 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:11.937277 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:11.939055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8148509a79e8420692d8d139e10a6cf9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.939731 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8148509a79e8420692d8d139e10a6cf9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.947371 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8148509a79e8420692d8d139e10a6cf9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.948017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8148509a79e8420692d8d139e10a6cf9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:11.966918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1344/2704] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:11 2022] DELETE /identity/v3/users/8148509a79e8420692d8d139e10a6cf9 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:12.006706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:12.006706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:12.016276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1361/2705] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:12.120669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:12.122223 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:12.191293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daddf7-988a-4b1f-b573-587a327c9d64 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:12.196625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:12.200335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:12.200335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:12.200335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] PATH_INFO: `/v3/users/2623b90df9c142858e88d30da1826f87` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:12.200335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2623b90df9c142858e88d30da1826f87)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:12.201035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:12.202881 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:12.394607 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2623b90df9c142858e88d30da1826f87'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.409667 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:12.412828 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:12.426402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f507a9d-cff1-4962-b0dc-8793b4ce7fae service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:12.431356 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:12.432061 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:12.432351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:12.432627 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] PATH_INFO: `/v3/projects/920f3da073f2481f95d11af8ad7bb390` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:12.433762 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=920f3da073f2481f95d11af8ad7bb390)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:12.435691 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:12.438467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:12.471090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2623b90df9c142858e88d30da1826f87'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.471734 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2623b90df9c142858e88d30da1826f87'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.479679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2623b90df9c142858e88d30da1826f87'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.480394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b93256a4-2ac3-449d-9046-0c7c2aa09a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2623b90df9c142858e88d30da1826f87'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.497450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1345/2706] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:12 2022] DELETE /identity/v3/users/2623b90df9c142858e88d30da1826f87 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:12.501922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad5e3-8d4d-4bd8-b0bf-da5a7caf0f21 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:12.502157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad5e3-8d4d-4bd8-b0bf-da5a7caf0f21 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:12.502432 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad5e3-8d4d-4bd8-b0bf-da5a7caf0f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:12.521937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-965ad5e3-8d4d-4bd8-b0bf-da5a7caf0f21 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:12.522980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-965ad5e3-8d4d-4bd8-b0bf-da5a7caf0f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:12.561733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '920f3da073f2481f95d11af8ad7bb390', 'request_id': 'req-a81ba0b1-b954-463e-9316-c16d9a71f949'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.588147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1346/2707] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:12.598864 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ad5e3-8d4d-4bd8-b0bf-da5a7caf0f21 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:12.600283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-965ad5e3-8d4d-4bd8-b0bf-da5a7caf0f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:12.659479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ad5e3-8d4d-4bd8-b0bf-da5a7caf0f21 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:12.661733 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '920f3da073f2481f95d11af8ad7bb390', 'request_id': 'req-a81ba0b1-b954-463e-9316-c16d9a71f949'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.670851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '920f3da073f2481f95d11af8ad7bb390', 'request_id': 'req-a81ba0b1-b954-463e-9316-c16d9a71f949'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.684215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] The token cache is being invalidate because project 920f3da073f2481f95d11af8ad7bb390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:12.684851 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.685750 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-a81ba0b1-b954-463e-9316-c16d9a71f949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:12.688199 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1362/2708] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:12 2022] DELETE /identity/v3/projects/920f3da073f2481f95d11af8ad7bb390 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:12.694934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920f0c8-361e-4ab1-935f-a2fd02c66cdf None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:12.694934 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920f0c8-361e-4ab1-935f-a2fd02c66cdf None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:12.695463 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920f0c8-361e-4ab1-935f-a2fd02c66cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:12.716235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:12.718026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1920f0c8-361e-4ab1-935f-a2fd02c66cdf None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:12.718152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:12.718152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:12.718152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:12.718405 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1920f0c8-361e-4ab1-935f-a2fd02c66cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:12.720989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:12.794505 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1363/2709] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:12.795989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c465-b8c6-431c-ac0e-80fbc1c09abb None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:12.796280 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c465-b8c6-431c-ac0e-80fbc1c09abb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:12.796553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c465-b8c6-431c-ac0e-80fbc1c09abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:12.800438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:12.801917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:12.812940 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1347/2710] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:57:12.820609 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-be63c465-b8c6-431c-ac0e-80fbc1c09abb None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:12.822225 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-be63c465-b8c6-431c-ac0e-80fbc1c09abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:12.831198 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1364/2711] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:12.835454 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:12.846932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc33d-7f29-413c-9e48-ff2b87976cef service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:12.852335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:12.853091 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:12.853375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:12.853645 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:12.854367 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:57:12.854669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:12.856283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:12.871090 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1348/2712] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:57:12.875589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63c465-b8c6-431c-ac0e-80fbc1c09abb None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:12.885759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63c465-b8c6-431c-ac0e-80fbc1c09abb None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:12.891187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bbb33-fdca-4967-a41d-31a0484080e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:12.893022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bbb33-fdca-4967-a41d-31a0484080e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:12.893022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bbb33-fdca-4967-a41d-31a0484080e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:12.893370 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bbb33-fdca-4967-a41d-31a0484080e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:12.893913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bbb33-fdca-4967-a41d-31a0484080e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:12.895855 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bbb33-fdca-4967-a41d-31a0484080e2 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.004905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1365/2713] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:57:12 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:13.007737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.019631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361952b1-bd26-468e-aa79-56e0bea80b07 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.082148 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b90372-6bcc-419b-90cb-4e0440007502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.083025 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90372-6bcc-419b-90cb-4e0440007502 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.083389 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90372-6bcc-419b-90cb-4e0440007502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.083727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90372-6bcc-419b-90cb-4e0440007502 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.084426 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b90372-6bcc-419b-90cb-4e0440007502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.086460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b90372-6bcc-419b-90cb-4e0440007502 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.194911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1349/2714] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:57:13 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:13.199384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bbb33-fdca-4967-a41d-31a0484080e2 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.209973 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bbb33-fdca-4967-a41d-31a0484080e2 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.214796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.215776 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.215992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.216253 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.216909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.218591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.231183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1366/2715] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:13.237993 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b90372-6bcc-419b-90cb-4e0440007502 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.248453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b90372-6bcc-419b-90cb-4e0440007502 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.254390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3702613-b21d-4288-bbf3-7e478338d359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.255344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3702613-b21d-4288-bbf3-7e478338d359 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.255625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3702613-b21d-4288-bbf3-7e478338d359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.255899 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3702613-b21d-4288-bbf3-7e478338d359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.256650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3702613-b21d-4288-bbf3-7e478338d359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.258283 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3702613-b21d-4288-bbf3-7e478338d359 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.265324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.267482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:13.271235 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1350/2716] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:13.275140 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3702613-b21d-4288-bbf3-7e478338d359 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.282336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda9585-3070-42a3-9cbd-dc3c4ef7c2c3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.287481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.287657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3702613-b21d-4288-bbf3-7e478338d359 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.288124 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.288586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.288857 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] PATH_INFO: `/v3/projects/06d78afb0fa541f996a8446bd684dbf4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.290285 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d78afb0fa541f996a8446bd684dbf4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.292316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.293477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517bc3d-facd-45a3-8d2a-1fac292cc736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.294070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517bc3d-facd-45a3-8d2a-1fac292cc736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.294364 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517bc3d-facd-45a3-8d2a-1fac292cc736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.294768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517bc3d-facd-45a3-8d2a-1fac292cc736 admin admin] PATH_INFO: `/v3/projects/ff8e2098f8b5496d91049ee0e3b33304/users/e0e4af2f951044d28cdb42715815b11d/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.311681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517bc3d-facd-45a3-8d2a-1fac292cc736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8e2098f8b5496d91049ee0e3b33304, user_id=e0e4af2f951044d28cdb42715815b11d, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.315047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517bc3d-facd-45a3-8d2a-1fac292cc736 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.345658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:13.450988 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1351/2717] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:13 2022] PUT /identity/v3/projects/ff8e2098f8b5496d91049ee0e3b33304/users/e0e4af2f951044d28cdb42715815b11d/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:57:13.463639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647bdb5-abd3-46e9-998d-6ab7293a2639 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.463922 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647bdb5-abd3-46e9-998d-6ab7293a2639 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.464196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647bdb5-abd3-46e9-998d-6ab7293a2639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.504538 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-4647bdb5-abd3-46e9-998d-6ab7293a2639 None None] MFA Rules not processed for user `e0e4af2f951044d28cdb42715815b11d`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:13.505790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4647bdb5-abd3-46e9-998d-6ab7293a2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:13.513218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d78afb0fa541f996a8446bd684dbf4', 'request_id': 'req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:13.574948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1352/2718] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:13.575879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647bdb5-abd3-46e9-998d-6ab7293a2639 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.577293 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4647bdb5-abd3-46e9-998d-6ab7293a2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:13.633587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d78afb0fa541f996a8446bd684dbf4', 'request_id': 'req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:13.637373 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647bdb5-abd3-46e9-998d-6ab7293a2639 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.641696 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fc7cc-351f-4ec7-9281-c625277902a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.642682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc7cc-351f-4ec7-9281-c625277902a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.643045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc7cc-351f-4ec7-9281-c625277902a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.643386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc7cc-351f-4ec7-9281-c625277902a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.644312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d78afb0fa541f996a8446bd684dbf4', 'request_id': 'req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:13.645331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3d8fc7cc-351f-4ec7-9281-c625277902a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:13.659705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] The token cache is being invalidate because project 06d78afb0fa541f996a8446bd684dbf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:13.660351 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:13.661056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:13.663380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1367/2719] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:13 2022] DELETE /identity/v3/projects/06d78afb0fa541f996a8446bd684dbf4 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:57:13.664265 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.665705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:13.718122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fc7cc-351f-4ec7-9281-c625277902a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.719579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fc7cc-351f-4ec7-9281-c625277902a9 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.724908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9528-c4d5-42a7-b60c-b16bbe47254e admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.728802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f76e0-18a5-435b-9992-9b8fcefa052e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d874536170494496769b4f37234b2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.729521 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f76e0-18a5-435b-9992-9b8fcefa052e service glance] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.729803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f76e0-18a5-435b-9992-9b8fcefa052e service glance] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.730110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f76e0-18a5-435b-9992-9b8fcefa052e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.730962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1353/2720] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:13.732213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-618f76e0-18a5-435b-9992-9b8fcefa052e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:13.732449 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f403f-4c8a-4f51-8d5c-1d9763e453ab None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.732898 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f403f-4c8a-4f51-8d5c-1d9763e453ab None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.733185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f403f-4c8a-4f51-8d5c-1d9763e453ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.745805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f76e0-18a5-435b-9992-9b8fcefa052e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.747286 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f76e0-18a5-435b-9992-9b8fcefa052e service glance] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.755622 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-0d8f403f-4c8a-4f51-8d5c-1d9763e453ab None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:13.757087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0d8f403f-4c8a-4f51-8d5c-1d9763e453ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:13.763075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1368/2721] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:13.823522 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1354/2722] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:13.831830 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce0b8f-6fc8-4671-949f-1f3a67d28ec6 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.832142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce0b8f-6fc8-4671-949f-1f3a67d28ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.832477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce0b8f-6fc8-4671-949f-1f3a67d28ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.864739 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-79ce0b8f-6fc8-4671-949f-1f3a67d28ec6 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:13.865994 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-79ce0b8f-6fc8-4671-949f-1f3a67d28ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:13.874528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1369/2723] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:13.879951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f403f-4c8a-4f51-8d5c-1d9763e453ab None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.890995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f403f-4c8a-4f51-8d5c-1d9763e453ab None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.902187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.902187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.902187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.902187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.902536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:57:13.902536 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.903832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:13.917190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1355/2724] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:57:13.921547 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce0b8f-6fc8-4671-949f-1f3a67d28ec6 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:13.932954 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce0b8f-6fc8-4671-949f-1f3a67d28ec6 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:13.937978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd2dfc-549b-4782-b0dd-7d684624d062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:13.938590 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd2dfc-549b-4782-b0dd-7d684624d062 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:13.939738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd2dfc-549b-4782-b0dd-7d684624d062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:13.940045 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd2dfc-549b-4782-b0dd-7d684624d062 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:13.940829 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd2dfc-549b-4782-b0dd-7d684624d062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:13.942862 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd2dfc-549b-4782-b0dd-7d684624d062 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:14.124070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1370/2725] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:57:13 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:57:14.129006 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:14.139251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534e424-d892-451b-90a0-b3a6762815f8 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:14.201512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a787df-3fbf-4c86-8c40-b4bf122a8cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:14.202154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a787df-3fbf-4c86-8c40-b4bf122a8cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:14.202508 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a787df-3fbf-4c86-8c40-b4bf122a8cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:14.202799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a787df-3fbf-4c86-8c40-b4bf122a8cbd admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:14.203409 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a787df-3fbf-4c86-8c40-b4bf122a8cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:14.205358 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a787df-3fbf-4c86-8c40-b4bf122a8cbd admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:14.325119 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1356/2726] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:57:14 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:14.330631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd2dfc-549b-4782-b0dd-7d684624d062 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:14.346123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd2dfc-549b-4782-b0dd-7d684624d062 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:14.351246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c737753-1e81-47be-9d91-9928afb6a211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:14.351963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737753-1e81-47be-9d91-9928afb6a211 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:14.352245 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737753-1e81-47be-9d91-9928afb6a211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:14.352520 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737753-1e81-47be-9d91-9928afb6a211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:14.353168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c737753-1e81-47be-9d91-9928afb6a211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:14.354803 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c737753-1e81-47be-9d91-9928afb6a211 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:14.368786 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1371/2727] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:14.373151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a787df-3fbf-4c86-8c40-b4bf122a8cbd admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:14.384338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a787df-3fbf-4c86-8c40-b4bf122a8cbd admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:14.389820 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dab70f-c51d-4e05-be66-f302057f1371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:14.393424 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dab70f-c51d-4e05-be66-f302057f1371 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:14.393680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dab70f-c51d-4e05-be66-f302057f1371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:14.393990 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dab70f-c51d-4e05-be66-f302057f1371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:14.394656 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dab70f-c51d-4e05-be66-f302057f1371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:14.396314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dab70f-c51d-4e05-be66-f302057f1371 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:14.408859 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1357/2728] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:57:14.413442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c737753-1e81-47be-9d91-9928afb6a211 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:14.424992 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c737753-1e81-47be-9d91-9928afb6a211 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:14.429672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:14.430361 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:14.430642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:14.430919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] PATH_INFO: `/v3/projects/8dc26bb88784480687b5e8bc3980d81f/users/88c0ad3e99004d09861b9b0e9d5bea77/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:14.446485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc26bb88784480687b5e8bc3980d81f, user_id=88c0ad3e99004d09861b9b0e9d5bea77, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:14.449259 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:14.601195 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:14.606866 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1372/2729] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:14 2022] PUT /identity/v3/projects/8dc26bb88784480687b5e8bc3980d81f/users/88c0ad3e99004d09861b9b0e9d5bea77/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:14.619199 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e498e-5fd9-48b8-961a-2359441564d7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:14.619487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e498e-5fd9-48b8-961a-2359441564d7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:14.619766 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e498e-5fd9-48b8-961a-2359441564d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:14.665465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-dd4e498e-5fd9-48b8-961a-2359441564d7 None None] MFA Rules not processed for user `88c0ad3e99004d09861b9b0e9d5bea77`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:14.667213 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-dd4e498e-5fd9-48b8-961a-2359441564d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:14.736438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1358/2730] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:14.748028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:14.749909 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:14.808966 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb65925-4cf1-4981-81c0-24df2f9b375b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:14.812637 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee85f8-9339-4da0-bb89-251ff5a3b3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:14.817557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee85f8-9339-4da0-bb89-251ff5a3b3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:14.817557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee85f8-9339-4da0-bb89-251ff5a3b3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:14.817557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee85f8-9339-4da0-bb89-251ff5a3b3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:14.823895 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee85f8-9339-4da0-bb89-251ff5a3b3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:14.825341 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee85f8-9339-4da0-bb89-251ff5a3b3a7 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:14.834215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1373/2731] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:15.440322 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:16.219486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=100 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:18.875976 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e498e-5fd9-48b8-961a-2359441564d7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:18.877447 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-dd4e498e-5fd9-48b8-961a-2359441564d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:18.961135 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e498e-5fd9-48b8-961a-2359441564d7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:18.966033 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:18.966680 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:18.966964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:18.967238 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] PATH_INFO: `/v3/users/d5b709ce4f04423c8d589a6b9a833179` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:18.968379 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b709ce4f04423c8d589a6b9a833179)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:18.970361 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:18.973826 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:19.045794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee85f8-9339-4da0-bb89-251ff5a3b3a7 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:19.058299 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee85f8-9339-4da0-bb89-251ff5a3b3a7 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:19.063264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:19.064955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:19.064955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:19.064955 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] PATH_INFO: `/v3/users/88c0ad3e99004d09861b9b0e9d5bea77` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:19.066567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c0ad3e99004d09861b9b0e9d5bea77)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:19.068528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:19.082951 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:19.148567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b709ce4f04423c8d589a6b9a833179'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.235947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b709ce4f04423c8d589a6b9a833179'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.236593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b709ce4f04423c8d589a6b9a833179'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.244744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b709ce4f04423c8d589a6b9a833179'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.245398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c827971c-7b5b-4491-9a3b-eecbe194ee13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b709ce4f04423c8d589a6b9a833179'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.261796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c0ad3e99004d09861b9b0e9d5bea77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.274748 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1359/2732] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:18 2022] DELETE /identity/v3/users/d5b709ce4f04423c8d589a6b9a833179 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:19.277725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c0ad3e99004d09861b9b0e9d5bea77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.279302 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845df4d-feae-4a15-b3ef-1bdced3a03af None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:19.279582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845df4d-feae-4a15-b3ef-1bdced3a03af None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:19.279858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845df4d-feae-4a15-b3ef-1bdced3a03af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:19.300132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a845df4d-feae-4a15-b3ef-1bdced3a03af None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:19.301704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a845df4d-feae-4a15-b3ef-1bdced3a03af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:19.365792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1360/2733] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:19.367587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c0ad3e99004d09861b9b0e9d5bea77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.369004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c0ad3e99004d09861b9b0e9d5bea77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.369799 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cbd0ea59-d808-4543-971f-331f57d6175e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c0ad3e99004d09861b9b0e9d5bea77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:19.375978 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845df4d-feae-4a15-b3ef-1bdced3a03af None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:19.377822 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a845df4d-feae-4a15-b3ef-1bdced3a03af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:19.387183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1374/2734] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:19 2022] DELETE /identity/v3/users/88c0ad3e99004d09861b9b0e9d5bea77 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:57:19.391877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67246760-84a4-41a4-994a-70d032e6a217 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:19.392181 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67246760-84a4-41a4-994a-70d032e6a217 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:19.392451 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67246760-84a4-41a4-994a-70d032e6a217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:19.413155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-67246760-84a4-41a4-994a-70d032e6a217 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:19.414411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-67246760-84a4-41a4-994a-70d032e6a217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:19.441355 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845df4d-feae-4a15-b3ef-1bdced3a03af None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:19.482838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:19.487396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1375/2735] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:19.497927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67246760-84a4-41a4-994a-70d032e6a217 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:19.499327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-67246760-84a4-41a4-994a-70d032e6a217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:19.500932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:19.501582 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:19.501873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:19.502185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:19.504719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:19.512819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67246760-84a4-41a4-994a-70d032e6a217 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:19.516923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:19.517552 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:19.517670 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:19.517833 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:19.518144 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:19.520116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:19.528544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:19.530028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:19.531017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1361/2736] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:57:19.540933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1376/2737] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:57:20.143818 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:20.145226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:20.157991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b46d6-b288-478e-b8d1-0bd1d1a3997a service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:20.163631 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:20.164785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:20.164785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:20.165271 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] PATH_INFO: `/v3/projects/a7e1eeabfb73438998aa1b8ffbbef75a` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:20.166947 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e1eeabfb73438998aa1b8ffbbef75a)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:20.168999 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:20.293398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e1eeabfb73438998aa1b8ffbbef75a', 'request_id': 'req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.354998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:20.357325 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:20.373356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e1eeabfb73438998aa1b8ffbbef75a', 'request_id': 'req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.376047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b10c56-9dc2-484c-93e7-0096961784c7 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:20.380921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e1eeabfb73438998aa1b8ffbbef75a', 'request_id': 'req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.406716 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] The token cache is being invalidate because project a7e1eeabfb73438998aa1b8ffbbef75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:20.411190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.411190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.411505 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1362/2738] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:20 2022] DELETE /identity/v3/projects/a7e1eeabfb73438998aa1b8ffbbef75a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:20.416425 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:20.421417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:20.455170 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:20.455972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:20.456710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:20.456710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] PATH_INFO: `/v3/projects/8dc26bb88784480687b5e8bc3980d81f` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:20.457742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc26bb88784480687b5e8bc3980d81f)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:20.459790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:20.488264 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:20.507203 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b434bc-b375-491c-bc05-bb57cd4bfbd9 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:20.511903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:20.512623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:20.512977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:20.513261 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] PATH_INFO: `/v3/users/34aa5d3c3ac7414fb8a800757f40bbc9` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:20.514440 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] RBAC: Authorizing `identity:delete_user(user_id=34aa5d3c3ac7414fb8a800757f40bbc9)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:20.516937 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:20.600907 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc26bb88784480687b5e8bc3980d81f', 'request_id': 'req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.740526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc26bb88784480687b5e8bc3980d81f', 'request_id': 'req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.748089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc26bb88784480687b5e8bc3980d81f', 'request_id': 'req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.762661 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] The token cache is being invalidate because project 8dc26bb88784480687b5e8bc3980d81f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:20.762915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.763693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d6c1beda-6b2f-42bb-bebe-9e60e92ace21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.765477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1377/2739] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:20 2022] DELETE /identity/v3/projects/8dc26bb88784480687b5e8bc3980d81f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:57:20.770412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1c638-6cbc-45f1-b533-36ed899d92c3 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:20.770704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1c638-6cbc-45f1-b533-36ed899d92c3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:20.770985 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1c638-6cbc-45f1-b533-36ed899d92c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:20.791736 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-70b1c638-6cbc-45f1-b533-36ed899d92c3 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:20.792949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-70b1c638-6cbc-45f1-b533-36ed899d92c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:20.856302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1378/2740] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:20.865584 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7725a-00fa-4403-b3fa-cb11f91ed857 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:20.866010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7725a-00fa-4403-b3fa-cb11f91ed857 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:20.866268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7725a-00fa-4403-b3fa-cb11f91ed857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:20.886058 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34aa5d3c3ac7414fb8a800757f40bbc9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:20.891889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-1fb7725a-00fa-4403-b3fa-cb11f91ed857 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:20.893300 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-1fb7725a-00fa-4403-b3fa-cb11f91ed857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:20.901749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1379/2741] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:20.906431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7725a-00fa-4403-b3fa-cb11f91ed857 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:20.917110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7725a-00fa-4403-b3fa-cb11f91ed857 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:20.922067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:20.922742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:20.923028 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:20.923266 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:20.923917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:57:20.924212 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:20.926482 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:20.938442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1380/2742] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:57:20.944128 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:20.954316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624d47b-7e01-44bf-a6fd-b71400370488 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:20.959479 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82619c5-1516-4872-b2ff-1111c943edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:20.960094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82619c5-1516-4872-b2ff-1111c943edcf admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:20.960345 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82619c5-1516-4872-b2ff-1111c943edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:20.960612 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82619c5-1516-4872-b2ff-1111c943edcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:20.961227 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82619c5-1516-4872-b2ff-1111c943edcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:20.963143 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82619c5-1516-4872-b2ff-1111c943edcf admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:21.073792 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34aa5d3c3ac7414fb8a800757f40bbc9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.074639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34aa5d3c3ac7414fb8a800757f40bbc9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.083032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34aa5d3c3ac7414fb8a800757f40bbc9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.083686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34aa5d3c3ac7414fb8a800757f40bbc9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.085791 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:21.104753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1363/2743] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:20 2022] DELETE /identity/v3/users/34aa5d3c3ac7414fb8a800757f40bbc9 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:21.114758 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1381/2744] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:57:20 2022] POST /identity/v3/projects => generated 391 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:57:21.118851 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:21.120554 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:21.318226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc31e8e-4adc-4f8b-a01d-356aaa0743df admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:21.386138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690bd9e-8afc-4e7a-92a2-c58fa5587790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:21.386753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690bd9e-8afc-4e7a-92a2-c58fa5587790 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:21.387026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690bd9e-8afc-4e7a-92a2-c58fa5587790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:21.387295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690bd9e-8afc-4e7a-92a2-c58fa5587790 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:21.387918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690bd9e-8afc-4e7a-92a2-c58fa5587790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:21.390237 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690bd9e-8afc-4e7a-92a2-c58fa5587790 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:21.465430 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1364/2745] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:57:21 2022] POST /identity/v3/users => generated 296 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:21.469771 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82619c5-1516-4872-b2ff-1111c943edcf admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:21.484223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82619c5-1516-4872-b2ff-1111c943edcf admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:21.489511 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9f47e-6120-4f44-869c-0dadc2c9eaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:21.490149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9f47e-6120-4f44-869c-0dadc2c9eaab admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:21.490421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9f47e-6120-4f44-869c-0dadc2c9eaab admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:21.490692 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9f47e-6120-4f44-869c-0dadc2c9eaab admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:21.491335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9f47e-6120-4f44-869c-0dadc2c9eaab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:21.492998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9f47e-6120-4f44-869c-0dadc2c9eaab admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:21.504090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1382/2746] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:21.510165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690bd9e-8afc-4e7a-92a2-c58fa5587790 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:21.520955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690bd9e-8afc-4e7a-92a2-c58fa5587790 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:21.525648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:21.526272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:21.526550 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:21.526819 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:21.527456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:21.529045 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:21.540800 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1365/2747] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:21.545125 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9f47e-6120-4f44-869c-0dadc2c9eaab admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:21.555747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:21.558009 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:21.561096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9f47e-6120-4f44-869c-0dadc2c9eaab admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:21.567921 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170c282-e2fe-446f-bb1d-262aec08a629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:21.568738 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170c282-e2fe-446f-bb1d-262aec08a629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:21.570272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170c282-e2fe-446f-bb1d-262aec08a629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:21.570272 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170c282-e2fe-446f-bb1d-262aec08a629 admin admin] PATH_INFO: `/v3/projects/f9c723fb54d643e59a3aa88815429443/users/974447a84a5847bd8f7b171765ab3fd6/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:21.574284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eede4f-a339-4057-827b-8a0800d71b12 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:21.581712 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:21.582458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:21.582740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:21.583183 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] PATH_INFO: `/v3/projects/84f268654e67490da489a09e32ff6da8` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:21.584438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f268654e67490da489a09e32ff6da8)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:21.586602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:21.589099 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170c282-e2fe-446f-bb1d-262aec08a629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c723fb54d643e59a3aa88815429443, user_id=974447a84a5847bd8f7b171765ab3fd6, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:21.596075 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170c282-e2fe-446f-bb1d-262aec08a629 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:21.697642 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1383/2748] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:21 2022] PUT /identity/v3/projects/f9c723fb54d643e59a3aa88815429443/users/974447a84a5847bd8f7b171765ab3fd6/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:21.708423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dd917-e524-45a5-9b13-7519ea0afbf3 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:21.708753 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dd917-e524-45a5-9b13-7519ea0afbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:21.709090 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dd917-e524-45a5-9b13-7519ea0afbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:21.733104 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f268654e67490da489a09e32ff6da8', 'request_id': 'req-6969f6df-290a-4614-8cb0-a21469bd5c7c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.740083 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-4d8dd917-e524-45a5-9b13-7519ea0afbf3 None None] MFA Rules not processed for user `974447a84a5847bd8f7b171765ab3fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:21.741334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4d8dd917-e524-45a5-9b13-7519ea0afbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:21.760945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:21.793243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f268654e67490da489a09e32ff6da8', 'request_id': 'req-6969f6df-290a-4614-8cb0-a21469bd5c7c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.799089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:21.801512 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f268654e67490da489a09e32ff6da8', 'request_id': 'req-6969f6df-290a-4614-8cb0-a21469bd5c7c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.810010 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1384/2749] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:21.814460 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] The token cache is being invalidate because project 84f268654e67490da489a09e32ff6da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:21.814759 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.815661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:21.817362 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1366/2750] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:21 2022] DELETE /identity/v3/projects/84f268654e67490da489a09e32ff6da8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:57:21.821117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dd917-e524-45a5-9b13-7519ea0afbf3 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:21.821297 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:21.822602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:21.822755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4d8dd917-e524-45a5-9b13-7519ea0afbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:21.887972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dd917-e524-45a5-9b13-7519ea0afbf3 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:21.891899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:21.892505 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:21.892782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:21.892891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969f6df-290a-4614-8cb0-a21469bd5c7c admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:21.893060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:21.895344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:21.898154 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:21.898779 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:21.899055 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:21.899325 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] PATH_INFO: `/v3/users/ad0682650b0142df97b138deeb64da02` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:21.900456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0682650b0142df97b138deeb64da02)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:21.902438 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:21.912378 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:21.914093 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:21.926811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1385/2751] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:21 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:22.124461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0682650b0142df97b138deeb64da02'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.186957 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0682650b0142df97b138deeb64da02'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.187591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0682650b0142df97b138deeb64da02'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.196873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0682650b0142df97b138deeb64da02'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.197560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-e52b3a17-0e5a-405e-8471-3d887998fd93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0682650b0142df97b138deeb64da02'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.216608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1367/2752] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:21 2022] DELETE /identity/v3/users/ad0682650b0142df97b138deeb64da02 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:22.641288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:22.652972 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7e41a-08b4-4d00-812a-bbd62d799906 service cinder] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:22.717262 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:22.717928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:22.718213 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:22.718487 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] PATH_INFO: `/v3/projects/b1ea4fb1269d40a2a4931ef1c85269fa` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:22.719646 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ea4fb1269d40a2a4931ef1c85269fa)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:22.721595 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:22.839402 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ea4fb1269d40a2a4931ef1c85269fa', 'request_id': 'req-75c738b3-b7b4-48d0-8584-26e02ac1291e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.927745 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ea4fb1269d40a2a4931ef1c85269fa', 'request_id': 'req-75c738b3-b7b4-48d0-8584-26e02ac1291e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.935153 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ea4fb1269d40a2a4931ef1c85269fa', 'request_id': 'req-75c738b3-b7b4-48d0-8584-26e02ac1291e'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.939219 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:22.946490 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] The token cache is being invalidate because project b1ea4fb1269d40a2a4931ef1c85269fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:22.946789 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.947464 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-75c738b3-b7b4-48d0-8584-26e02ac1291e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:22.948997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1386/2753] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:22 2022] DELETE /identity/v3/projects/b1ea4fb1269d40a2a4931ef1c85269fa => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:22.954042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23b1e6-41c9-4aab-bfab-350b39aa76c8 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:22.954525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23b1e6-41c9-4aab-bfab-350b39aa76c8 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:22.954525 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23b1e6-41c9-4aab-bfab-350b39aa76c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:22.976918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-ee23b1e6-41c9-4aab-bfab-350b39aa76c8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:22.978219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ee23b1e6-41c9-4aab-bfab-350b39aa76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:23.042588 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1368/2754] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:23.050775 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061a4d8-2c23-4293-83b4-a9c8081c3047 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.051065 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061a4d8-2c23-4293-83b4-a9c8081c3047 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.051389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061a4d8-2c23-4293-83b4-a9c8081c3047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.071923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3061a4d8-2c23-4293-83b4-a9c8081c3047 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:23.076963 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3061a4d8-2c23-4293-83b4-a9c8081c3047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:23.083978 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1387/2755] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:23.088307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23b1e6-41c9-4aab-bfab-350b39aa76c8 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:23.102556 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23b1e6-41c9-4aab-bfab-350b39aa76c8 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:23.107344 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:23.107960 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.108242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.108520 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.109177 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:57:23.109507 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:23.111145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:23.123570 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:23.126857 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1369/2756] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:57:23.132428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061a4d8-2c23-4293-83b4-a9c8081c3047 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:23.146089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061a4d8-2c23-4293-83b4-a9c8081c3047 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:23.149805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ff83-269a-4c5a-b536-f577a4425369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:23.150478 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ff83-269a-4c5a-b536-f577a4425369 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.150756 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ff83-269a-4c5a-b536-f577a4425369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.151035 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ff83-269a-4c5a-b536-f577a4425369 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.151701 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03ff83-269a-4c5a-b536-f577a4425369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:23.153610 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03ff83-269a-4c5a-b536-f577a4425369 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:23.265425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1388/2757] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:57:23 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:23.269421 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:23.279677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91332-0bce-45c4-9276-a406bb3f5383 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:23.337688 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826fe82-2a9e-4a29-8dae-1c0ed33b0c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:23.338347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826fe82-2a9e-4a29-8dae-1c0ed33b0c52 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.338660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826fe82-2a9e-4a29-8dae-1c0ed33b0c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.338945 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826fe82-2a9e-4a29-8dae-1c0ed33b0c52 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.339585 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826fe82-2a9e-4a29-8dae-1c0ed33b0c52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:23.341762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826fe82-2a9e-4a29-8dae-1c0ed33b0c52 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:23.427887 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1370/2758] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:57:23 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:23.432133 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ff83-269a-4c5a-b536-f577a4425369 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:23.444958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ff83-269a-4c5a-b536-f577a4425369 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:23.449743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18caf5-fba1-4286-a53a-fb043f7e70a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:23.450390 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18caf5-fba1-4286-a53a-fb043f7e70a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.450675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18caf5-fba1-4286-a53a-fb043f7e70a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.450953 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18caf5-fba1-4286-a53a-fb043f7e70a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.451606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18caf5-fba1-4286-a53a-fb043f7e70a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:23.453205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18caf5-fba1-4286-a53a-fb043f7e70a8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:23.465317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1389/2759] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:23.469519 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826fe82-2a9e-4a29-8dae-1c0ed33b0c52 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:23.479956 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826fe82-2a9e-4a29-8dae-1c0ed33b0c52 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:23.484752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa8f9-d889-4873-aa65-b4254d164354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:23.485396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa8f9-d889-4873-aa65-b4254d164354 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.485679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa8f9-d889-4873-aa65-b4254d164354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.485989 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa8f9-d889-4873-aa65-b4254d164354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.486670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa8f9-d889-4873-aa65-b4254d164354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:23.489188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa8f9-d889-4873-aa65-b4254d164354 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:23.499711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1371/2760] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:23.504626 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18caf5-fba1-4286-a53a-fb043f7e70a8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:23.515079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18caf5-fba1-4286-a53a-fb043f7e70a8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:23.521038 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:23.521694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.522008 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.522288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] PATH_INFO: `/v3/projects/e3f7d4f1194747d2873e0df87f5640fb/users/28a01fb159dd4097a5277423b178c3e0/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.539304 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3f7d4f1194747d2873e0df87f5640fb, user_id=28a01fb159dd4097a5277423b178c3e0, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:23.542060 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:23.644711 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1390/2761] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:23 2022] PUT /identity/v3/projects/e3f7d4f1194747d2873e0df87f5640fb/users/28a01fb159dd4097a5277423b178c3e0/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:23.656719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e072c-1cca-4e4f-a8e7-4a40f1700612 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.657749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e072c-1cca-4e4f-a8e7-4a40f1700612 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.657749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e072c-1cca-4e4f-a8e7-4a40f1700612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.690096 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-8d3e072c-1cca-4e4f-a8e7-4a40f1700612 None None] MFA Rules not processed for user `28a01fb159dd4097a5277423b178c3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:23.696642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8d3e072c-1cca-4e4f-a8e7-4a40f1700612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:23.761608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1372/2762] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:23.773944 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:23.775677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:23.838860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c3b2-6e59-40ad-9e93-9fb2b0aab03a admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:23.843690 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad06b0c-d929-41a8-94e3-3125690b7c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:23.844302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad06b0c-d929-41a8-94e3-3125690b7c2e service cinder] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:23.844576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad06b0c-d929-41a8-94e3-3125690b7c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:23.845255 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad06b0c-d929-41a8-94e3-3125690b7c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:23.854216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad06b0c-d929-41a8-94e3-3125690b7c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:23.855622 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad06b0c-d929-41a8-94e3-3125690b7c2e service cinder] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:23.865654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1391/2763] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:29.190846 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e072c-1cca-4e4f-a8e7-4a40f1700612 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:29.253136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e072c-1cca-4e4f-a8e7-4a40f1700612 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:29.257906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:29.258531 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:29.258809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:29.259078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] PATH_INFO: `/v3/users/28a01fb159dd4097a5277423b178c3e0` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:29.260190 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a01fb159dd4097a5277423b178c3e0)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:29.262121 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:29.267817 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:29.490219 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a01fb159dd4097a5277423b178c3e0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:29.555888 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a01fb159dd4097a5277423b178c3e0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:29.556510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a01fb159dd4097a5277423b178c3e0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:29.564034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a01fb159dd4097a5277423b178c3e0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:29.564691 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a01fb159dd4097a5277423b178c3e0'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:29.583913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1373/2764] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:29 2022] DELETE /identity/v3/users/28a01fb159dd4097a5277423b178c3e0 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:29.588705 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d269-113a-4549-a04e-ace0a3e613d1 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:29.589115 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d269-113a-4549-a04e-ace0a3e613d1 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:29.589415 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d269-113a-4549-a04e-ace0a3e613d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:29.611846 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-5615d269-113a-4549-a04e-ace0a3e613d1 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:29.613202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5615d269-113a-4549-a04e-ace0a3e613d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:29.674495 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1392/2765] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:57:29.684396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:29.685874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:29.766848 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bde5-55e8-4b10-9455-4e5f32e78bd4 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:29.771184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c144a-b867-4ff0-840e-d53ac187481c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:29.771802 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c144a-b867-4ff0-840e-d53ac187481c service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:29.772088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c144a-b867-4ff0-840e-d53ac187481c service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:29.772359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c144a-b867-4ff0-840e-d53ac187481c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:29.781646 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c144a-b867-4ff0-840e-d53ac187481c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:29.783129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c144a-b867-4ff0-840e-d53ac187481c service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:29.793420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1374/2766] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:30.194445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615d269-113a-4549-a04e-ace0a3e613d1 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:30.195884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5615d269-113a-4549-a04e-ace0a3e613d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:30.208431 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615d269-113a-4549-a04e-ace0a3e613d1 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:30.213886 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40559361-3623-4c19-b328-802b4770514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:30.214121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40559361-3623-4c19-b328-802b4770514c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:30.214427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40559361-3623-4c19-b328-802b4770514c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:30.214702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40559361-3623-4c19-b328-802b4770514c admin admin] PATH_INFO: `/v3/projects/e3f7d4f1194747d2873e0df87f5640fb` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:30.215860 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40559361-3623-4c19-b328-802b4770514c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3f7d4f1194747d2873e0df87f5640fb)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:30.217874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40559361-3623-4c19-b328-802b4770514c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:30.309180 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:30.324704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-40559361-3623-4c19-b328-802b4770514c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3f7d4f1194747d2873e0df87f5640fb', 'request_id': 'req-40559361-3623-4c19-b328-802b4770514c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:30.427110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-40559361-3623-4c19-b328-802b4770514c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3f7d4f1194747d2873e0df87f5640fb', 'request_id': 'req-40559361-3623-4c19-b328-802b4770514c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:30.434006 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-40559361-3623-4c19-b328-802b4770514c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3f7d4f1194747d2873e0df87f5640fb', 'request_id': 'req-40559361-3623-4c19-b328-802b4770514c'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:30.445417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-40559361-3623-4c19-b328-802b4770514c admin admin] The token cache is being invalidate because project e3f7d4f1194747d2873e0df87f5640fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:30.445752 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-40559361-3623-4c19-b328-802b4770514c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:30.446513 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-40559361-3623-4c19-b328-802b4770514c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:30.448372 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1393/2767] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:30 2022] DELETE /identity/v3/projects/e3f7d4f1194747d2873e0df87f5640fb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:30.453524 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9b6a9-5dba-460a-b838-4ccce2bf9ce4 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:30.457337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9b6a9-5dba-460a-b838-4ccce2bf9ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:30.457337 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9b6a9-5dba-460a-b838-4ccce2bf9ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:30.474214 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-fea9b6a9-5dba-460a-b838-4ccce2bf9ce4 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:30.475480 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-fea9b6a9-5dba-460a-b838-4ccce2bf9ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:30.538844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1375/2768] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:30.603109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d513dd-bf2b-4974-ab2c-992043f6a17c None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:30.603677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d513dd-bf2b-4974-ab2c-992043f6a17c None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:30.603677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d513dd-bf2b-4974-ab2c-992043f6a17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:30.624305 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-73d513dd-bf2b-4974-ab2c-992043f6a17c None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:30.625528 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-73d513dd-bf2b-4974-ab2c-992043f6a17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:30.634336 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1394/2769] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:30.638606 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9b6a9-5dba-460a-b838-4ccce2bf9ce4 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:30.649166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9b6a9-5dba-460a-b838-4ccce2bf9ce4 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:30.654332 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:30.654936 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:30.655218 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:30.655492 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:30.656178 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:57:30.656511 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:30.658168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:30.661879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:30.671542 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1376/2770] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:57:30.676488 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d513dd-bf2b-4974-ab2c-992043f6a17c None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:30.690638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d513dd-bf2b-4974-ab2c-992043f6a17c None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:30.696142 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626fdbc-528c-4853-a19e-c8f47fe534eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:30.696770 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626fdbc-528c-4853-a19e-c8f47fe534eb admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:30.697055 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626fdbc-528c-4853-a19e-c8f47fe534eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:30.697327 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626fdbc-528c-4853-a19e-c8f47fe534eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:30.698001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626fdbc-528c-4853-a19e-c8f47fe534eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:30.699889 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626fdbc-528c-4853-a19e-c8f47fe534eb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:30.803946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1395/2771] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:57:30 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:30.808341 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:30.818892 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c3e4a-8be7-4789-aabf-100ecedab9b4 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:30.876189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010870db-2a0f-450b-bf76-dd89c4bbbab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:30.876689 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010870db-2a0f-450b-bf76-dd89c4bbbab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:30.876968 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010870db-2a0f-450b-bf76-dd89c4bbbab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:30.877241 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010870db-2a0f-450b-bf76-dd89c4bbbab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:30.877891 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010870db-2a0f-450b-bf76-dd89c4bbbab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:30.879934 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010870db-2a0f-450b-bf76-dd89c4bbbab6 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:30.994345 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1377/2772] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:57:30 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:57:30.998923 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626fdbc-528c-4853-a19e-c8f47fe534eb admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:31.010839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626fdbc-528c-4853-a19e-c8f47fe534eb admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:31.015983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e36ca-d95f-494c-a858-be8d13ada2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:31.016587 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e36ca-d95f-494c-a858-be8d13ada2cb admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:31.016899 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e36ca-d95f-494c-a858-be8d13ada2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:31.017328 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e36ca-d95f-494c-a858-be8d13ada2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:31.018077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e36ca-d95f-494c-a858-be8d13ada2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:31.019897 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e36ca-d95f-494c-a858-be8d13ada2cb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:31.030413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1396/2773] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:31.035079 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010870db-2a0f-450b-bf76-dd89c4bbbab6 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:31.045428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010870db-2a0f-450b-bf76-dd89c4bbbab6 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:31.050807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cb6be-6b22-493e-a867-eb8813ba9d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:31.051417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cb6be-6b22-493e-a867-eb8813ba9d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:31.051700 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cb6be-6b22-493e-a867-eb8813ba9d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:31.051977 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cb6be-6b22-493e-a867-eb8813ba9d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:31.052639 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cb6be-6b22-493e-a867-eb8813ba9d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:31.054249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cb6be-6b22-493e-a867-eb8813ba9d8e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:31.064950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1378/2774] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:57:31.069232 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e36ca-d95f-494c-a858-be8d13ada2cb admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:31.079505 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e36ca-d95f-494c-a858-be8d13ada2cb admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:31.084912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:31.086534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:31.086534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:31.086534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] PATH_INFO: `/v3/projects/97f94240e9764a2eaa3d22d3a3860d1c/users/bdb257a487b7453bbd8c50df0358ba41/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:31.102154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f94240e9764a2eaa3d22d3a3860d1c, user_id=bdb257a487b7453bbd8c50df0358ba41, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:31.104835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:31.211209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1397/2775] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:57:31 2022] PUT /identity/v3/projects/97f94240e9764a2eaa3d22d3a3860d1c/users/bdb257a487b7453bbd8c50df0358ba41/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:31.221195 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581a0b5-dc86-481f-97ab-79241964e4da None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:31.221475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581a0b5-dc86-481f-97ab-79241964e4da None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:31.221745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581a0b5-dc86-481f-97ab-79241964e4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:31.249919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-9581a0b5-dc86-481f-97ab-79241964e4da None None] MFA Rules not processed for user `bdb257a487b7453bbd8c50df0358ba41`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:31.251124 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9581a0b5-dc86-481f-97ab-79241964e4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:31.314951 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1379/2776] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:31.327256 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:31.329166 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:31.388077 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1ece8-2cc4-4ef6-98cc-d363cb3ec1ef admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:31.392001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329e3a3-d3c0-4443-aca8-f974b3f34567 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d874536170494496769b4f37234b2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:31.392603 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329e3a3-d3c0-4443-aca8-f974b3f34567 service glance] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:31.392884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329e3a3-d3c0-4443-aca8-f974b3f34567 service glance] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:31.393158 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329e3a3-d3c0-4443-aca8-f974b3f34567 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:31.402268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329e3a3-d3c0-4443-aca8-f974b3f34567 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:31.403702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329e3a3-d3c0-4443-aca8-f974b3f34567 service glance] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:31.413557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1398/2777] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:41.312187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:41.408744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:46.268328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581a0b5-dc86-481f-97ab-79241964e4da None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:46.270099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-9581a0b5-dc86-481f-97ab-79241964e4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:46.352528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581a0b5-dc86-481f-97ab-79241964e4da None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:46.358267 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:46.358900 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:46.359185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:46.359455 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] PATH_INFO: `/v3/users/974447a84a5847bd8f7b171765ab3fd6` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:46.360629 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] RBAC: Authorizing `identity:delete_user(user_id=974447a84a5847bd8f7b171765ab3fd6)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:46.362598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:46.636942 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '974447a84a5847bd8f7b171765ab3fd6'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:46.727310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '974447a84a5847bd8f7b171765ab3fd6'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:46.728132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '974447a84a5847bd8f7b171765ab3fd6'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:46.735966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '974447a84a5847bd8f7b171765ab3fd6'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:46.736686 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '974447a84a5847bd8f7b171765ab3fd6'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:46.763150 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1380/2778] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:57:46 2022] DELETE /identity/v3/users/974447a84a5847bd8f7b171765ab3fd6 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:46.768000 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311c46-31af-4b68-813b-08742c7e2dee None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:46.768564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311c46-31af-4b68-813b-08742c7e2dee None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:46.768564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311c46-31af-4b68-813b-08742c7e2dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:46.789861 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3e311c46-31af-4b68-813b-08742c7e2dee None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:46.791112 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3e311c46-31af-4b68-813b-08742c7e2dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:46.854214 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1399/2779] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:46.864443 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:46.866098 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:46.950162 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c143e89-cb5f-437d-b859-c939a0a4576f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:46.953931 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:46.954599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:46.954880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:46.955086 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:46.967711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:46.969165 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:46.980730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1381/2780] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:47.520835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e311c46-31af-4b68-813b-08742c7e2dee None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:47.523507 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3e311c46-31af-4b68-813b-08742c7e2dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:47.536986 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e311c46-31af-4b68-813b-08742c7e2dee None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:47.542002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:47.542665 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:47.542946 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:47.546459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] PATH_INFO: `/v3/projects/f9c723fb54d643e59a3aa88815429443` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:47.546459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c723fb54d643e59a3aa88815429443)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:47.547419 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:47.688903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c723fb54d643e59a3aa88815429443', 'request_id': 'req-5b2954ce-8311-4adf-bc5d-c2cecd565c74'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:47.843990 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c723fb54d643e59a3aa88815429443', 'request_id': 'req-5b2954ce-8311-4adf-bc5d-c2cecd565c74'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:47.854922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c723fb54d643e59a3aa88815429443', 'request_id': 'req-5b2954ce-8311-4adf-bc5d-c2cecd565c74'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:47.873021 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] The token cache is being invalidate because project f9c723fb54d643e59a3aa88815429443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:57:47.873306 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:47.874034 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-5b2954ce-8311-4adf-bc5d-c2cecd565c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:57:47.875559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1400/2781] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:57:47 2022] DELETE /identity/v3/projects/f9c723fb54d643e59a3aa88815429443 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:57:48.979205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:48.980807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:49.008465 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:49.065735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63383e52-df6c-4e7c-8dc3-0f2114452253 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:49.121624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:49.122619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:49.122995 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:49.123276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:49.125238 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:49.216093 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:49.217537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:49.228406 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1382/2782] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:57.864439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:59.222871 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=67 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:59.797512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53cb52-9df9-4be7-ac4a-3cc0c2d9cead None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:59.798184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53cb52-9df9-4be7-ac4a-3cc0c2d9cead None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:59.798184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53cb52-9df9-4be7-ac4a-3cc0c2d9cead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:59.819443 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-ec53cb52-9df9-4be7-ac4a-3cc0c2d9cead None None] MFA Rules not processed for user `f19bf0e6a8ad425cbbf804d6d7be5a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:59.821391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ec53cb52-9df9-4be7-ac4a-3cc0c2d9cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:59.828718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1401/2783] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:57:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:03.423089 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:03.424605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:03.521535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1f0dc-107d-4441-97b4-71dd288ef7ec service cinder] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:03.526880 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ab837-797e-43e9-a148-9dc5db89351e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:03.529486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab837-797e-43e9-a148-9dc5db89351e service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:03.529486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab837-797e-43e9-a148-9dc5db89351e service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:03.529486 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab837-797e-43e9-a148-9dc5db89351e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:03.538885 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-833ab837-797e-43e9-a148-9dc5db89351e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:03.552463 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ab837-797e-43e9-a148-9dc5db89351e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:03.554193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ab837-797e-43e9-a148-9dc5db89351e service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:03.565649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1383/2784] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:04.570578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53cb52-9df9-4be7-ac4a-3cc0c2d9cead None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:04.574334 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-ec53cb52-9df9-4be7-ac4a-3cc0c2d9cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:04.679176 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53cb52-9df9-4be7-ac4a-3cc0c2d9cead None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:04.684710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:04.685369 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:04.685649 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:04.685941 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] PATH_INFO: `/v3/users/e0e4af2f951044d28cdb42715815b11d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:04.687107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e4af2f951044d28cdb42715815b11d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:04.689007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:04.929539 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e4af2f951044d28cdb42715815b11d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:04.943121 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e4af2f951044d28cdb42715815b11d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:05.006307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e4af2f951044d28cdb42715815b11d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:05.007042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e4af2f951044d28cdb42715815b11d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:05.007723 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e4af2f951044d28cdb42715815b11d'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:05.029764 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1402/2785] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:04 2022] DELETE /identity/v3/users/e0e4af2f951044d28cdb42715815b11d => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:05.034595 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa404-bb5e-49fa-8ef8-f51bf0bc551d None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:05.035122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa404-bb5e-49fa-8ef8-f51bf0bc551d None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:05.035210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa404-bb5e-49fa-8ef8-f51bf0bc551d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:05.056487 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a5bfa404-bb5e-49fa-8ef8-f51bf0bc551d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:05.057710 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a5bfa404-bb5e-49fa-8ef8-f51bf0bc551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:05.124314 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1384/2786] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:05.134835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:05.140565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:05.195448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91858f37-c54e-4926-812e-10fe7f3b33cf admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:05.199224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d96e4-7e7a-49bf-9d95-2ef8459bb1bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:05.199876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d96e4-7e7a-49bf-9d95-2ef8459bb1bc service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:05.200107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d96e4-7e7a-49bf-9d95-2ef8459bb1bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:05.200386 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d96e4-7e7a-49bf-9d95-2ef8459bb1bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:05.209767 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d96e4-7e7a-49bf-9d95-2ef8459bb1bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:05.211239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d96e4-7e7a-49bf-9d95-2ef8459bb1bc service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:05.220983 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1403/2787] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:05.707711 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfa404-bb5e-49fa-8ef8-f51bf0bc551d None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:05.709187 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a5bfa404-bb5e-49fa-8ef8-f51bf0bc551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:05.723625 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfa404-bb5e-49fa-8ef8-f51bf0bc551d None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:05.739399 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:05.740653 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:05.741411 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:05.742028 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] PATH_INFO: `/v3/projects/ff8e2098f8b5496d91049ee0e3b33304` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:05.744029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8e2098f8b5496d91049ee0e3b33304)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:05.746877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:05.897365 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8e2098f8b5496d91049ee0e3b33304', 'request_id': 'req-8009cf74-4d8a-4489-81ee-9ee113b89cc9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:06.008911 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8e2098f8b5496d91049ee0e3b33304', 'request_id': 'req-8009cf74-4d8a-4489-81ee-9ee113b89cc9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:06.017080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8e2098f8b5496d91049ee0e3b33304', 'request_id': 'req-8009cf74-4d8a-4489-81ee-9ee113b89cc9'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:06.020077 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:06.029965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] The token cache is being invalidate because project ff8e2098f8b5496d91049ee0e3b33304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:06.030265 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:06.031717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-8009cf74-4d8a-4489-81ee-9ee113b89cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:06.032921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1385/2788] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:05 2022] DELETE /identity/v3/projects/ff8e2098f8b5496d91049ee0e3b33304 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:06.038241 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f136a-1072-4f68-9c28-a58715305ea8 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.038755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f136a-1072-4f68-9c28-a58715305ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.038852 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f136a-1072-4f68-9c28-a58715305ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.061446 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a57f136a-1072-4f68-9c28-a58715305ea8 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:06.062706 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a57f136a-1072-4f68-9c28-a58715305ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:06.074056 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:06.126848 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1404/2789] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:06.136207 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2f89-9a80-4188-8e73-63eaab6915af None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.143561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2f89-9a80-4188-8e73-63eaab6915af None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.143561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2f89-9a80-4188-8e73-63eaab6915af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.158356 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-13ac2f89-9a80-4188-8e73-63eaab6915af None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:06.159490 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-13ac2f89-9a80-4188-8e73-63eaab6915af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:06.168185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1386/2790] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:58:06.173872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f136a-1072-4f68-9c28-a58715305ea8 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:06.184400 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f136a-1072-4f68-9c28-a58715305ea8 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:06.189732 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:06.190549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.191193 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.191805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.192211 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:58:06.192823 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:06.194797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:06.207796 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1405/2791] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:58:06.212796 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac2f89-9a80-4188-8e73-63eaab6915af None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:06.222703 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac2f89-9a80-4188-8e73-63eaab6915af None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:06.228422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbf565-087b-4f79-967d-d5794df5b1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:06.229054 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbf565-087b-4f79-967d-d5794df5b1fe admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.229333 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbf565-087b-4f79-967d-d5794df5b1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.229683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbf565-087b-4f79-967d-d5794df5b1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.230419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbf565-087b-4f79-967d-d5794df5b1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:06.232392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbf565-087b-4f79-967d-d5794df5b1fe admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:06.335303 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1387/2792] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:06 2022] POST /identity/v3/projects => generated 397 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:06.339606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:06.350483 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59408832-1b41-4798-a845-0b8601a5594d admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:06.419553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84c771-1521-47cc-a90c-47f9827b281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:06.420289 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84c771-1521-47cc-a90c-47f9827b281c admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.420573 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84c771-1521-47cc-a90c-47f9827b281c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.420922 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84c771-1521-47cc-a90c-47f9827b281c admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.421834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84c771-1521-47cc-a90c-47f9827b281c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:06.423850 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84c771-1521-47cc-a90c-47f9827b281c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:06.514004 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1406/2793] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:06 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:06.515205 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbf565-087b-4f79-967d-d5794df5b1fe admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:06.527044 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbf565-087b-4f79-967d-d5794df5b1fe admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:06.532567 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef8131-e7bc-4da4-9b37-f34a6a95cced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:06.538762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef8131-e7bc-4da4-9b37-f34a6a95cced admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.538762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef8131-e7bc-4da4-9b37-f34a6a95cced admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.538762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef8131-e7bc-4da4-9b37-f34a6a95cced admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.538762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef8131-e7bc-4da4-9b37-f34a6a95cced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:06.538762 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef8131-e7bc-4da4-9b37-f34a6a95cced admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:06.548864 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1388/2794] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:58:06.552957 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84c771-1521-47cc-a90c-47f9827b281c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:06.564790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84c771-1521-47cc-a90c-47f9827b281c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:06.570069 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1a66d-bdba-486c-b249-1e85edc0f7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:06.578177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a66d-bdba-486c-b249-1e85edc0f7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.578177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a66d-bdba-486c-b249-1e85edc0f7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.578177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a66d-bdba-486c-b249-1e85edc0f7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.578177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1a66d-bdba-486c-b249-1e85edc0f7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:06.578177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1a66d-bdba-486c-b249-1e85edc0f7c6 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:06.585811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1407/2795] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:58:06.590485 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef8131-e7bc-4da4-9b37-f34a6a95cced admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:06.601321 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef8131-e7bc-4da4-9b37-f34a6a95cced admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:06.607732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:06.607732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.607980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.608255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] PATH_INFO: `/v3/projects/b3950807726642f7a73516124aa34890/users/845a7bff20724681924b2de73153bc77/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.624876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3950807726642f7a73516124aa34890, user_id=845a7bff20724681924b2de73153bc77, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:06.627553 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:06.744812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1389/2796] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:06 2022] PUT /identity/v3/projects/b3950807726642f7a73516124aa34890/users/845a7bff20724681924b2de73153bc77/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:06.762268 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db5fe1-0ff5-4359-b81f-2e31da3f02d0 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.762839 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db5fe1-0ff5-4359-b81f-2e31da3f02d0 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.763470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db5fe1-0ff5-4359-b81f-2e31da3f02d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.794109 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a4db5fe1-0ff5-4359-b81f-2e31da3f02d0 None None] MFA Rules not processed for user `845a7bff20724681924b2de73153bc77`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:06.795349 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a4db5fe1-0ff5-4359-b81f-2e31da3f02d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:06.869312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1408/2797] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:06.881133 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:06.883035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:06.946838 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3848c-b21e-496c-9ef0-9d0fb37e5913 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:06.950725 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99824c1c-7a10-4dbc-b4e8-9bae351aaaca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:06.951326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99824c1c-7a10-4dbc-b4e8-9bae351aaaca service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:06.951602 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99824c1c-7a10-4dbc-b4e8-9bae351aaaca service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:06.951868 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99824c1c-7a10-4dbc-b4e8-9bae351aaaca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:06.960471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99824c1c-7a10-4dbc-b4e8-9bae351aaaca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:06.963035 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99824c1c-7a10-4dbc-b4e8-9bae351aaaca service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:06.973926 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1390/2798] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:09.593675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db5fe1-0ff5-4359-b81f-2e31da3f02d0 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:09.595261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a4db5fe1-0ff5-4359-b81f-2e31da3f02d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:09.608577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:09.662436 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db5fe1-0ff5-4359-b81f-2e31da3f02d0 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:09.667401 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:09.668018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:09.668296 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:09.668568 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] PATH_INFO: `/v3/users/bdb257a487b7453bbd8c50df0358ba41` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:09.669878 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb257a487b7453bbd8c50df0358ba41)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:09.672398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:09.852183 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb257a487b7453bbd8c50df0358ba41'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:09.864592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb257a487b7453bbd8c50df0358ba41'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:09.928413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb257a487b7453bbd8c50df0358ba41'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:09.929078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb257a487b7453bbd8c50df0358ba41'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:09.929698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb257a487b7453bbd8c50df0358ba41'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:09.943910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1409/2799] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:09 2022] DELETE /identity/v3/users/bdb257a487b7453bbd8c50df0358ba41 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:09.949295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7fe2-66a5-4ba0-82d7-dc8b2435dbea None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:09.949844 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7fe2-66a5-4ba0-82d7-dc8b2435dbea None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:09.949987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7fe2-66a5-4ba0-82d7-dc8b2435dbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:09.969273 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-170a7fe2-66a5-4ba0-82d7-dc8b2435dbea None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:09.970528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-170a7fe2-66a5-4ba0-82d7-dc8b2435dbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:10.030650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1391/2800] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:10.040438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:10.042178 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:10.101467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8c77e-5a07-46d7-a3a3-dacbeed2e6a8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:10.105559 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98037162-e3d3-4467-a483-26912416b273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:10.106192 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98037162-e3d3-4467-a483-26912416b273 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:10.106475 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98037162-e3d3-4467-a483-26912416b273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:10.106743 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98037162-e3d3-4467-a483-26912416b273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:10.115898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98037162-e3d3-4467-a483-26912416b273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:10.117309 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98037162-e3d3-4467-a483-26912416b273 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:10.127702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1410/2801] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:10.517032 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7fe2-66a5-4ba0-82d7-dc8b2435dbea None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:10.518496 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-170a7fe2-66a5-4ba0-82d7-dc8b2435dbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:10.531026 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7fe2-66a5-4ba0-82d7-dc8b2435dbea None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:10.536042 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:10.536681 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:10.536958 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:10.537231 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] PATH_INFO: `/v3/projects/97f94240e9764a2eaa3d22d3a3860d1c` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:10.538419 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f94240e9764a2eaa3d22d3a3860d1c)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:10.540335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:10.631477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:10.650545 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f94240e9764a2eaa3d22d3a3860d1c', 'request_id': 'req-7d637975-b7c4-4e6d-90e1-0307203627e3'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:10.725919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f94240e9764a2eaa3d22d3a3860d1c', 'request_id': 'req-7d637975-b7c4-4e6d-90e1-0307203627e3'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:10.733011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f94240e9764a2eaa3d22d3a3860d1c', 'request_id': 'req-7d637975-b7c4-4e6d-90e1-0307203627e3'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:10.745678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] The token cache is being invalidate because project 97f94240e9764a2eaa3d22d3a3860d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:10.745997 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:10.746789 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-7d637975-b7c4-4e6d-90e1-0307203627e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:10.748608 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1392/2802] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:10 2022] DELETE /identity/v3/projects/97f94240e9764a2eaa3d22d3a3860d1c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:10.753903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7892b-df36-4f03-92bc-ab5f9570bcae None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:10.754397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7892b-df36-4f03-92bc-ab5f9570bcae None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:10.754512 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7892b-df36-4f03-92bc-ab5f9570bcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:10.774589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-3cb7892b-df36-4f03-92bc-ab5f9570bcae None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:10.776720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3cb7892b-df36-4f03-92bc-ab5f9570bcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:10.807524 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:10.840456 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1411/2803] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:58:10.849338 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4d0c-56d3-43b3-85c9-94903d398302 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:10.849634 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4d0c-56d3-43b3-85c9-94903d398302 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:10.849952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4d0c-56d3-43b3-85c9-94903d398302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:10.871075 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-a90c4d0c-56d3-43b3-85c9-94903d398302 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:10.872230 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a90c4d0c-56d3-43b3-85c9-94903d398302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:10.880149 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1393/2804] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:10.884474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7892b-df36-4f03-92bc-ab5f9570bcae None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:10.895107 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7892b-df36-4f03-92bc-ab5f9570bcae None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:10.899884 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:10.901239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:10.901624 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:10.901928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:10.903901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:58:10.903901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:10.907746 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:10.919474 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1412/2805] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:58:10.924070 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c4d0c-56d3-43b3-85c9-94903d398302 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:10.934804 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c4d0c-56d3-43b3-85c9-94903d398302 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:10.939642 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cf0c5-1d5f-4643-9c93-996ba807a12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:10.940249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cf0c5-1d5f-4643-9c93-996ba807a12f admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:10.940535 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cf0c5-1d5f-4643-9c93-996ba807a12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:10.940809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cf0c5-1d5f-4643-9c93-996ba807a12f admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:10.941458 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cf0c5-1d5f-4643-9c93-996ba807a12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:10.943410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cf0c5-1d5f-4643-9c93-996ba807a12f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:11.049950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1394/2806] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:10 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:11.054190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:11.064919 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d93daf-f919-4b3f-8241-615a98122a33 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:11.125815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea7a30-c58e-424f-90ef-e20fb42f12ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:11.126502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea7a30-c58e-424f-90ef-e20fb42f12ab admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:11.126790 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea7a30-c58e-424f-90ef-e20fb42f12ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:11.127070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea7a30-c58e-424f-90ef-e20fb42f12ab admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:11.127699 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea7a30-c58e-424f-90ef-e20fb42f12ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:11.129606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea7a30-c58e-424f-90ef-e20fb42f12ab admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:11.203414 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1413/2807] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:11 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:11.207755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cf0c5-1d5f-4643-9c93-996ba807a12f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:11.219420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cf0c5-1d5f-4643-9c93-996ba807a12f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:11.224136 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ee358-f825-4125-95ca-d06538396f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:11.224746 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee358-f825-4125-95ca-d06538396f87 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:11.225024 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee358-f825-4125-95ca-d06538396f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:11.225300 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee358-f825-4125-95ca-d06538396f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:11.226016 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ee358-f825-4125-95ca-d06538396f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:11.227586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ee358-f825-4125-95ca-d06538396f87 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:11.237909 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1395/2808] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:11.242135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea7a30-c58e-424f-90ef-e20fb42f12ab admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:11.252645 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea7a30-c58e-424f-90ef-e20fb42f12ab admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:11.257430 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570df90-5663-4fb7-acb8-ac408d3f8c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:11.258136 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570df90-5663-4fb7-acb8-ac408d3f8c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:11.258416 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570df90-5663-4fb7-acb8-ac408d3f8c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:11.258688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570df90-5663-4fb7-acb8-ac408d3f8c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:11.259874 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570df90-5663-4fb7-acb8-ac408d3f8c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:11.261529 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570df90-5663-4fb7-acb8-ac408d3f8c22 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:11.272392 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1414/2809] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:11.277298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ee358-f825-4125-95ca-d06538396f87 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:11.287405 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ee358-f825-4125-95ca-d06538396f87 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:11.292226 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:11.292874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:11.293155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:11.293446 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] PATH_INFO: `/v3/projects/e76f79d4de9d4d548401bb7c2611c312/users/7c0a39e6411f47779d72e7c41c7c1cd7/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:11.308123 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76f79d4de9d4d548401bb7c2611c312, user_id=7c0a39e6411f47779d72e7c41c7c1cd7, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:11.310783 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:11.363870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1396/2810] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:11 2022] PUT /identity/v3/projects/e76f79d4de9d4d548401bb7c2611c312/users/7c0a39e6411f47779d72e7c41c7c1cd7/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:58:11.375114 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3e991-aaf2-416d-bb25-26b562988a58 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:11.375489 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3e991-aaf2-416d-bb25-26b562988a58 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:11.375834 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3e991-aaf2-416d-bb25-26b562988a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:11.404755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-94e3e991-aaf2-416d-bb25-26b562988a58 None None] MFA Rules not processed for user `7c0a39e6411f47779d72e7c41c7c1cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:11.406293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-94e3e991-aaf2-416d-bb25-26b562988a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:11.471514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1415/2811] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:11.482274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:11.483975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:11.542289 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6be21-3c15-4cd7-8a54-c1eec4f9f948 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:11.545964 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ef8c0-101d-4daa-97a8-2cd94e903095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7a064cf91243648c6cc6d6ca576505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:11.546577 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ef8c0-101d-4daa-97a8-2cd94e903095 service cinder] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:11.546959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ef8c0-101d-4daa-97a8-2cd94e903095 service cinder] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:11.547127 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ef8c0-101d-4daa-97a8-2cd94e903095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:11.555950 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ef8c0-101d-4daa-97a8-2cd94e903095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:11.558275 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ef8c0-101d-4daa-97a8-2cd94e903095 service cinder] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:11.567650 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1397/2812] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:16.032278 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3e991-aaf2-416d-bb25-26b562988a58 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.034149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-94e3e991-aaf2-416d-bb25-26b562988a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:16.067660 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:16.101154 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3e991-aaf2-416d-bb25-26b562988a58 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.106534 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.107215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.107481 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.107755 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] PATH_INFO: `/v3/users/845a7bff20724681924b2de73153bc77` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.108898 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] RBAC: Authorizing `identity:delete_user(user_id=845a7bff20724681924b2de73153bc77)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.110843 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.246892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '845a7bff20724681924b2de73153bc77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.259578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '845a7bff20724681924b2de73153bc77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.316709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '845a7bff20724681924b2de73153bc77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.317377 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '845a7bff20724681924b2de73153bc77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.318023 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '845a7bff20724681924b2de73153bc77'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.332403 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1416/2813] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:16 2022] DELETE /identity/v3/users/845a7bff20724681924b2de73153bc77 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:16.337317 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197772-ee99-4266-8bcd-e084d99a40dc None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.337906 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197772-ee99-4266-8bcd-e084d99a40dc None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.338111 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197772-ee99-4266-8bcd-e084d99a40dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.357730 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-c6197772-ee99-4266-8bcd-e084d99a40dc None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:16.359052 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c6197772-ee99-4266-8bcd-e084d99a40dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:16.411001 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:16.423003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1398/2814] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:16.433721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.435215 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:16.494875 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32fced-f911-4dcb-9178-e8ac0a51719f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.498577 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8c2b8-72b0-472a-907d-419f96e487e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.499187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8c2b8-72b0-472a-907d-419f96e487e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.499555 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8c2b8-72b0-472a-907d-419f96e487e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.499731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8c2b8-72b0-472a-907d-419f96e487e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.509591 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8c2b8-72b0-472a-907d-419f96e487e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.511757 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8c2b8-72b0-472a-907d-419f96e487e9 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.521819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1417/2815] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:17.142693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6197772-ee99-4266-8bcd-e084d99a40dc None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:17.144658 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-c6197772-ee99-4266-8bcd-e084d99a40dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:17.157913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6197772-ee99-4266-8bcd-e084d99a40dc None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:17.163113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:17.164256 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.164256 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.164471 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] PATH_INFO: `/v3/projects/b3950807726642f7a73516124aa34890` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.171210 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3950807726642f7a73516124aa34890)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:17.173155 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:17.287962 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3950807726642f7a73516124aa34890', 'request_id': 'req-ec568959-33d8-4f08-89c8-ed8efe842c7c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:17.342963 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3950807726642f7a73516124aa34890', 'request_id': 'req-ec568959-33d8-4f08-89c8-ed8efe842c7c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:17.349987 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3950807726642f7a73516124aa34890', 'request_id': 'req-ec568959-33d8-4f08-89c8-ed8efe842c7c'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:17.361920 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] The token cache is being invalidate because project b3950807726642f7a73516124aa34890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:17.362215 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:17.362870 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ec568959-33d8-4f08-89c8-ed8efe842c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:17.364394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1399/2816] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:17 2022] DELETE /identity/v3/projects/b3950807726642f7a73516124aa34890 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:17.370239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9643b73-4991-4dab-b99f-9918b2cfb47d None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.370847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9643b73-4991-4dab-b99f-9918b2cfb47d None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.370847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9643b73-4991-4dab-b99f-9918b2cfb47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.391417 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d9643b73-4991-4dab-b99f-9918b2cfb47d None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:17.392675 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d9643b73-4991-4dab-b99f-9918b2cfb47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:17.441190 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:17.457357 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1418/2817] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:58:17.465735 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea401-954c-4ac4-9c0e-ec768d230e6f None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.466074 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea401-954c-4ac4-9c0e-ec768d230e6f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.466609 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea401-954c-4ac4-9c0e-ec768d230e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.488305 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-e10ea401-954c-4ac4-9c0e-ec768d230e6f None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:17.489604 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-e10ea401-954c-4ac4-9c0e-ec768d230e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:17.498537 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1400/2818] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:17.503047 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9643b73-4991-4dab-b99f-9918b2cfb47d None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:17.514497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9643b73-4991-4dab-b99f-9918b2cfb47d None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:17.519496 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:17.519864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.520044 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.520314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.520962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81886) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:58:17.521261 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:17.523110 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:17.538741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1419/2819] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 17:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:58:17.539741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ea401-954c-4ac4-9c0e-ec768d230e6f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:17.549884 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ea401-954c-4ac4-9c0e-ec768d230e6f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:17.554664 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6437d-f327-481e-b657-1ec3d37a9373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:17.555284 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6437d-f327-481e-b657-1ec3d37a9373 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.555551 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6437d-f327-481e-b657-1ec3d37a9373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.555821 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6437d-f327-481e-b657-1ec3d37a9373 admin admin] PATH_INFO: `/v3/projects` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.556474 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6437d-f327-481e-b657-1ec3d37a9373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:17.559249 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6437d-f327-481e-b657-1ec3d37a9373 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:17.700445 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1401/2820] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:17 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:17.701988 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:17.712818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2dc4-809c-4f5d-bcc3-038e105b1cc3 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:17.776494 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2cc96-098a-4e1d-8777-e432bacc33db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:17.777108 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2cc96-098a-4e1d-8777-e432bacc33db admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.777380 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2cc96-098a-4e1d-8777-e432bacc33db admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.777647 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2cc96-098a-4e1d-8777-e432bacc33db admin admin] PATH_INFO: `/v3/users` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.778312 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2cc96-098a-4e1d-8777-e432bacc33db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:17.780200 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2cc96-098a-4e1d-8777-e432bacc33db admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:17.883820 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1420/2821] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:17 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:58:17.887326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6437d-f327-481e-b657-1ec3d37a9373 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:17.898255 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6437d-f327-481e-b657-1ec3d37a9373 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:17.903354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19525-e091-4482-a10d-cf83e3a38488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:17.904063 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19525-e091-4482-a10d-cf83e3a38488 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.904349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19525-e091-4482-a10d-cf83e3a38488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.904624 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19525-e091-4482-a10d-cf83e3a38488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.905276 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa19525-e091-4482-a10d-cf83e3a38488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:17.906914 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa19525-e091-4482-a10d-cf83e3a38488 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:17.918392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1402/2822] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:17.922486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2cc96-098a-4e1d-8777-e432bacc33db admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:17.933251 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2cc96-098a-4e1d-8777-e432bacc33db admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:17.938411 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09a21c-61af-47cc-a744-cf0675fd2b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:17.939012 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a21c-61af-47cc-a744-cf0675fd2b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.939292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a21c-61af-47cc-a744-cf0675fd2b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.939567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a21c-61af-47cc-a744-cf0675fd2b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.940221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09a21c-61af-47cc-a744-cf0675fd2b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:17.941815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09a21c-61af-47cc-a744-cf0675fd2b25 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:17.958397 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1421/2823] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:17.963078 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19525-e091-4482-a10d-cf83e3a38488 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:17.973359 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19525-e091-4482-a10d-cf83e3a38488 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:17.978239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:17.979842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:17.980122 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:17.980398 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] PATH_INFO: `/v3/projects/193c5ec786e74f138b2fc58ab35367ff/users/196ffd853b234fce845f359484d4b5e2/roles/fff136f214f2428895daf10483c542d1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:17.995913 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193c5ec786e74f138b2fc58ab35367ff, user_id=196ffd853b234fce845f359484d4b5e2, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:17.998579 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:18.081382 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1403/2824] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:17 2022] PUT /identity/v3/projects/193c5ec786e74f138b2fc58ab35367ff/users/196ffd853b234fce845f359484d4b5e2/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:18.093681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e3ef3-d8d8-48c1-adec-f6cce0e75bf7 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:18.094054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e3ef3-d8d8-48c1-adec-f6cce0e75bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:18.094440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e3ef3-d8d8-48c1-adec-f6cce0e75bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:18.122935 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-0d7e3ef3-d8d8-48c1-adec-f6cce0e75bf7 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:18.124260 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0d7e3ef3-d8d8-48c1-adec-f6cce0e75bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:18.216715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1422/2825] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:18.217328 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:18.218661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:18.284948 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cad442-24e3-4509-ba61-1bcc0ba5e7e4 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:18.288882 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:18.289528 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:18.289843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:18.290152 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:18.300635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:18.302144 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:18.312003 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1404/2826] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:20.989876 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e3ef3-d8d8-48c1-adec-f6cce0e75bf7 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:20.991316 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-0d7e3ef3-d8d8-48c1-adec-f6cce0e75bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:21.004293 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e3ef3-d8d8-48c1-adec-f6cce0e75bf7 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:21.007910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63e356-fe18-4cad-ac62-e873fca7afc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:21.008461 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63e356-fe18-4cad-ac62-e873fca7afc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:21.008741 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63e356-fe18-4cad-ac62-e873fca7afc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:21.009007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63e356-fe18-4cad-ac62-e873fca7afc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:21.011702 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-5f63e356-fe18-4cad-ac62-e873fca7afc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:21.073578 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:21.090651 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63e356-fe18-4cad-ac62-e873fca7afc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:21.092037 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63e356-fe18-4cad-ac62-e873fca7afc6 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:21.108374 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1423/2827] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:23.750690 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:23.754227 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:23.776742 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69acf9b7-8610-495c-bde2-23e3c8886a83 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:23.785088 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:23.786130 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:23.786559 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:23.786996 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] PATH_INFO: `/v3/users/7c0a39e6411f47779d72e7c41c7c1cd7` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:23.788331 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0a39e6411f47779d72e7c41c7c1cd7)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:23.792184 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:23.822917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:24.023917 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0a39e6411f47779d72e7c41c7c1cd7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:24.128031 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0a39e6411f47779d72e7c41c7c1cd7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:24.129250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0a39e6411f47779d72e7c41c7c1cd7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:24.142410 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0a39e6411f47779d72e7c41c7c1cd7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:24.143437 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0a39e6411f47779d72e7c41c7c1cd7'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:24.165678 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1405/2828] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:23 2022] DELETE /identity/v3/users/7c0a39e6411f47779d72e7c41c7c1cd7 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:24.172825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50ad9-48ae-4adf-9346-15ced9872fbf None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:24.173784 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50ad9-48ae-4adf-9346-15ced9872fbf None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:24.176288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50ad9-48ae-4adf-9346-15ced9872fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:24.200731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-efc50ad9-48ae-4adf-9346-15ced9872fbf None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:24.205516 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-efc50ad9-48ae-4adf-9346-15ced9872fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:24.290388 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1424/2829] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:58:24.305168 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:24.309172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:24.400874 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323b8c4-5ce9-47e1-9aa4-2ef39236c99d admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:24.428845 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12256d2f-9195-407a-9464-edf93188f149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:24.430418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12256d2f-9195-407a-9464-edf93188f149 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:24.430418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12256d2f-9195-407a-9464-edf93188f149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:24.431132 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12256d2f-9195-407a-9464-edf93188f149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:24.448787 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12256d2f-9195-407a-9464-edf93188f149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:24.450605 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12256d2f-9195-407a-9464-edf93188f149 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:24.461675 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1406/2830] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:25.115247 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc50ad9-48ae-4adf-9346-15ced9872fbf None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.117526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-efc50ad9-48ae-4adf-9346-15ced9872fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:25.161168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc50ad9-48ae-4adf-9346-15ced9872fbf None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.167664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.168466 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.168905 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.169423 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] PATH_INFO: `/v3/projects/e76f79d4de9d4d548401bb7c2611c312` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.170800 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76f79d4de9d4d548401bb7c2611c312)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.173239 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:25.309307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76f79d4de9d4d548401bb7c2611c312', 'request_id': 'req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.373116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529ebcd-98b8-4d3a-b986-7eab0e34a91f None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.373586 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529ebcd-98b8-4d3a-b986-7eab0e34a91f None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.373709 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529ebcd-98b8-4d3a-b986-7eab0e34a91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.403318 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-6529ebcd-98b8-4d3a-b986-7eab0e34a91f None None] MFA Rules not processed for user `196ffd853b234fce845f359484d4b5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:25.404670 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-6529ebcd-98b8-4d3a-b986-7eab0e34a91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:25.411847 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76f79d4de9d4d548401bb7c2611c312', 'request_id': 'req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.419317 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76f79d4de9d4d548401bb7c2611c312', 'request_id': 'req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.435498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] The token cache is being invalidate because project e76f79d4de9d4d548401bb7c2611c312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:25.435805 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.436480 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.449276 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1425/2831] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:25 2022] DELETE /identity/v3/projects/e76f79d4de9d4d548401bb7c2611c312 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:58:25.482813 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1407/2832] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:25.487816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.489406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:25.567564 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1f6ef-f6e0-4e5d-ace0-dafda637a268 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.572836 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c193c-b2bc-4e37-b2dd-ec52b7d3406b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.573704 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c193c-b2bc-4e37-b2dd-ec52b7d3406b admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.574015 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c193c-b2bc-4e37-b2dd-ec52b7d3406b admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.574291 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c193c-b2bc-4e37-b2dd-ec52b7d3406b admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.574937 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c193c-b2bc-4e37-b2dd-ec52b7d3406b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.576858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c193c-b2bc-4e37-b2dd-ec52b7d3406b admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:25.676156 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:25.684421 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1426/2833] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:25 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:25.688784 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529ebcd-98b8-4d3a-b986-7eab0e34a91f None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.701392 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529ebcd-98b8-4d3a-b986-7eab0e34a91f None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.721157 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:25.760065 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1ff32-2eb9-41f5-b1aa-d35be15c41a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.760684 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ff32-2eb9-41f5-b1aa-d35be15c41a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.760966 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ff32-2eb9-41f5-b1aa-d35be15c41a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.761239 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ff32-2eb9-41f5-b1aa-d35be15c41a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.761894 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1ff32-2eb9-41f5-b1aa-d35be15c41a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.764201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1ff32-2eb9-41f5-b1aa-d35be15c41a7 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:25.842741 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1408/2834] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:25 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:25.846798 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c193c-b2bc-4e37-b2dd-ec52b7d3406b admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.858106 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c193c-b2bc-4e37-b2dd-ec52b7d3406b admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.863059 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b35429-ac7c-4a99-8ee7-ec169fe2750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.863769 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b35429-ac7c-4a99-8ee7-ec169fe2750f admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.864054 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b35429-ac7c-4a99-8ee7-ec169fe2750f admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.864330 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b35429-ac7c-4a99-8ee7-ec169fe2750f admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.864982 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b35429-ac7c-4a99-8ee7-ec169fe2750f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.866686 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b35429-ac7c-4a99-8ee7-ec169fe2750f admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:25.881205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1427/2835] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:25.887196 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1ff32-2eb9-41f5-b1aa-d35be15c41a7 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.899598 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1ff32-2eb9-41f5-b1aa-d35be15c41a7 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.912649 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafa6a3-6fb6-46bf-be97-4f62d4bfa5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.913476 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafa6a3-6fb6-46bf-be97-4f62d4bfa5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.913812 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafa6a3-6fb6-46bf-be97-4f62d4bfa5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.914176 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafa6a3-6fb6-46bf-be97-4f62d4bfa5bb admin admin] PATH_INFO: `/v3/projects/97dcf1650ae04d40aa09583b93e4a8e9/users/813f01157f08489e8f602ba953cabd88/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.933017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafa6a3-6fb6-46bf-be97-4f62d4bfa5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dcf1650ae04d40aa09583b93e4a8e9, user_id=813f01157f08489e8f602ba953cabd88, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.935865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafa6a3-6fb6-46bf-be97-4f62d4bfa5bb admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.029699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1409/2836] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:25 2022] PUT /identity/v3/projects/97dcf1650ae04d40aa09583b93e4a8e9/users/813f01157f08489e8f602ba953cabd88/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:26.033664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b35429-ac7c-4a99-8ee7-ec169fe2750f admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.045026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b35429-ac7c-4a99-8ee7-ec169fe2750f admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.105592 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ed39c-016a-41de-baff-70643c38e8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.106149 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed39c-016a-41de-baff-70643c38e8db admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.106435 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed39c-016a-41de-baff-70643c38e8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.106709 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed39c-016a-41de-baff-70643c38e8db admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.107360 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ed39c-016a-41de-baff-70643c38e8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.108987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ed39c-016a-41de-baff-70643c38e8db admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.119096 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1428/2837] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:26.123461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafa6a3-6fb6-46bf-be97-4f62d4bfa5bb admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.133824 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafa6a3-6fb6-46bf-be97-4f62d4bfa5bb admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.139163 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.139828 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.140113 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.140386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] PATH_INFO: `/v3/projects/97dcf1650ae04d40aa09583b93e4a8e9/users/813f01157f08489e8f602ba953cabd88/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.142380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dcf1650ae04d40aa09583b93e4a8e9, user_id=813f01157f08489e8f602ba953cabd88, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.145011 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.156955 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1410/2838] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:26 2022] PUT /identity/v3/projects/97dcf1650ae04d40aa09583b93e4a8e9/users/813f01157f08489e8f602ba953cabd88/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:31.929804 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0793a-609d-4c6e-b1bb-f695900ea5a9 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:31.931715 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0793a-609d-4c6e-b1bb-f695900ea5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:31.932218 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0793a-609d-4c6e-b1bb-f695900ea5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:31.968462 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-a1e0793a-609d-4c6e-b1bb-f695900ea5a9 None None] MFA Rules not processed for user `813f01157f08489e8f602ba953cabd88`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:31.969677 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-a1e0793a-609d-4c6e-b1bb-f695900ea5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:32.046707 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1429/2839] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:58:32.059745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:32.061581 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:32.124745 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fddcb6-a9f6-4663-aa04-5cad1781c841 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:32.128833 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d874536170494496769b4f37234b2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:32.129459 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:32.129737 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:32.130073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:32.191050 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:32.192530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:32.202393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1411/2840] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:32.481229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72109-9873-4b41-b83e-5710e886adec None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:32.481517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72109-9873-4b41-b83e-5710e886adec None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:32.481794 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72109-9873-4b41-b83e-5710e886adec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:32.514221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-2bb72109-9873-4b41-b83e-5710e886adec None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:32.515535 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-2bb72109-9873-4b41-b83e-5710e886adec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:32.573730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:32.581182 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1430/2841] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:58:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:32.724683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:32.726185 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:32.762867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:32.804560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b910cf8-ff30-4807-ad92-a5df74f8fef1 service glance] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:32.810274 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94cbcc-9d67-41d5-95fc-b5372a8f2f0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:32.810908 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94cbcc-9d67-41d5-95fc-b5372a8f2f0b service swift] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:32.811188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94cbcc-9d67-41d5-95fc-b5372a8f2f0b service swift] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:32.811768 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94cbcc-9d67-41d5-95fc-b5372a8f2f0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:32.820193 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94cbcc-9d67-41d5-95fc-b5372a8f2f0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:32.821845 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94cbcc-9d67-41d5-95fc-b5372a8f2f0b service swift] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:32.830984 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1412/2842] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:58:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:58:42.827858 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=19 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:42.901335 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18c0e-a2ad-484f-b077-3ad275ff4d70 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:42.901903 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18c0e-a2ad-484f-b077-3ad275ff4d70 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:42.902002 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18c0e-a2ad-484f-b077-3ad275ff4d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:42.921337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-d9a18c0e-a2ad-484f-b077-3ad275ff4d70 None None] MFA Rules not processed for user `196ffd853b234fce845f359484d4b5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:42.922597 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d9a18c0e-a2ad-484f-b077-3ad275ff4d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:42.929774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1431/2843] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 17:58:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:46.320151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cb185-52de-49aa-8929-b49ec031e4f7 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:46.320704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cb185-52de-49aa-8929-b49ec031e4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:46.320704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cb185-52de-49aa-8929-b49ec031e4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:46.340853 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-547cb185-52de-49aa-8929-b49ec031e4f7 None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:46.342242 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-547cb185-52de-49aa-8929-b49ec031e4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:46.349652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1413/2844] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 17:58:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:46.370155 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a18c0e-a2ad-484f-b077-3ad275ff4d70 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:46.375439 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a18c0e-a2ad-484f-b077-3ad275ff4d70 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:46.380381 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab69a4-4200-4c1e-9d92-b74eb525182c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:46.381087 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab69a4-4200-4c1e-9d92-b74eb525182c service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:46.381425 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab69a4-4200-4c1e-9d92-b74eb525182c service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:46.381768 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab69a4-4200-4c1e-9d92-b74eb525182c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:46.392389 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab69a4-4200-4c1e-9d92-b74eb525182c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:46.394175 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab69a4-4200-4c1e-9d92-b74eb525182c service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:46.403168 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1432/2845] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 17:58:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:58:56.334979 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:56.399376 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=9 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:05.005509 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cb185-52de-49aa-8929-b49ec031e4f7 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:05.007087 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-547cb185-52de-49aa-8929-b49ec031e4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:05.066959 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cb185-52de-49aa-8929-b49ec031e4f7 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:05.070591 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:05.071201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:05.071482 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:05.072396 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:05.074294 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:05.157842 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:05.159310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:05.179720 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1414/2846] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:59:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:59:05.799233 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab69a4-4200-4c1e-9d92-b74eb525182c service swift] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:05.809668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab69a4-4200-4c1e-9d92-b74eb525182c service swift] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:05.813302 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:05.813989 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:05.814270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:05.814569 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:05.816323 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:05.830263 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:05.831724 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:05.842682 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1433/2847] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:59:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:59:15.177169 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=63 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:15.838209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=6 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:16.299323 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:16.309126 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc28c7-4439-4938-9384-9e4d4ce5d46a service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:16.312776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:16.313370 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:16.313641 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:16.313942 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:16.323683 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:16.325253 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:16.335080 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1415/2848] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:59:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:59:26.330786 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:39.206410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:39.218270 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492d51-afc6-4b86-b83a-32a35ce1465e service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:39.222363 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:39.222965 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:39.227398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:39.227398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:39.227398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:39.236406 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:39.237835 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:39.246858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1434/2849] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:59:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:59:49.242346 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=6 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:51.284243 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:51.285918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:51.305272 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162147fe-15f2-4908-9d10-dbe094b6079b service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:51.310998 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:51.312286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:51.312286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:51.312286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:51.313967 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:51.327980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:51.329632 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:51.342061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1416/2850] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 17:59:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:01.336116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:05.860079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.861634 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:05.872731 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fab6c4-381e-400d-a33e-98b75707bbb2 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.876638 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.877228 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.877526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.877793 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.879530 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:05.945224 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.947471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.957470 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1435/2851] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:09.499560 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:09.501138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:09.579295 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e48476-e0c1-4d25-a4f4-7b84a9dce3dc service nova] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:09.584493 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:09.585103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:09.585388 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:09.585669 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:09.587450 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:09.599647 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:09.601206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:09.612029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1417/2852] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:13.352319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:13.353818 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:13.368945 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b788d1-fdf6-4a45-938f-e6a09585e833 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:13.373962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:13.374586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:13.374863 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:13.376811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] PATH_INFO: `/v3/users/f19bf0e6a8ad425cbbf804d6d7be5a0a` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:13.376811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19bf0e6a8ad425cbbf804d6d7be5a0a)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:13.378164 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:13.565410 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19bf0e6a8ad425cbbf804d6d7be5a0a'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:13.578391 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19bf0e6a8ad425cbbf804d6d7be5a0a'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:13.655184 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19bf0e6a8ad425cbbf804d6d7be5a0a'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:13.655872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19bf0e6a8ad425cbbf804d6d7be5a0a'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:13.656567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19bf0e6a8ad425cbbf804d6d7be5a0a'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:13.669671 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1436/2853] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:13 2022] DELETE /identity/v3/users/f19bf0e6a8ad425cbbf804d6d7be5a0a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:13.972039 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:13.986220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396f9b7-86e0-4340-8861-f830411b6d5f service placement] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:14.050151 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:14.050790 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:14.051073 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:14.051349 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] PATH_INFO: `/v3/projects/54cebab3bf1f440dbaa4ef020177a4bb` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:14.052494 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=54cebab3bf1f440dbaa4ef020177a4bb)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:14.054442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:14.170138 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54cebab3bf1f440dbaa4ef020177a4bb', 'request_id': 'req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.243782 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54cebab3bf1f440dbaa4ef020177a4bb', 'request_id': 'req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.250439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54cebab3bf1f440dbaa4ef020177a4bb', 'request_id': 'req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.262721 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] The token cache is being invalidate because project 54cebab3bf1f440dbaa4ef020177a4bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:14.264543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.264543 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.265099 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1418/2854] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:13 2022] DELETE /identity/v3/projects/54cebab3bf1f440dbaa4ef020177a4bb => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:14.269091 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:14.270553 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:14.283792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bc327-6a20-4c21-96f4-7bc7a8d278f4 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:14.340872 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:14.341519 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:14.341778 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:14.342080 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] PATH_INFO: `/v3/users/b41cbf038c334a2fb1f7a73aebcf19c4` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:14.343216 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41cbf038c334a2fb1f7a73aebcf19c4)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:14.345132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:14.350662 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:14.624544 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41cbf038c334a2fb1f7a73aebcf19c4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.636815 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41cbf038c334a2fb1f7a73aebcf19c4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.723540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41cbf038c334a2fb1f7a73aebcf19c4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.724203 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41cbf038c334a2fb1f7a73aebcf19c4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.724940 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41cbf038c334a2fb1f7a73aebcf19c4'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:14.738913 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1437/2855] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:14 2022] DELETE /identity/v3/users/b41cbf038c334a2fb1f7a73aebcf19c4 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:14.984513 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:14.985971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:15.003549 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6710c-cb48-46d1-8ba5-d1860df73e0f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:15.022416 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:15.063640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:15.064278 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:15.064561 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:15.064839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] PATH_INFO: `/v3/projects/b93845e04a89497cb5f963884d95c2ba` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:15.066051 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93845e04a89497cb5f963884d95c2ba)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:15.067974 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:15.191368 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93845e04a89497cb5f963884d95c2ba', 'request_id': 'req-2be86fe0-f49a-446b-b316-d3fdef966475'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:15.253298 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93845e04a89497cb5f963884d95c2ba', 'request_id': 'req-2be86fe0-f49a-446b-b316-d3fdef966475'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:15.259648 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93845e04a89497cb5f963884d95c2ba', 'request_id': 'req-2be86fe0-f49a-446b-b316-d3fdef966475'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:15.271599 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] The token cache is being invalidate because project b93845e04a89497cb5f963884d95c2ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:15.271886 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:15.272526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-2be86fe0-f49a-446b-b316-d3fdef966475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:15.274034 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1419/2856] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:14 2022] DELETE /identity/v3/projects/b93845e04a89497cb5f963884d95c2ba => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:24.752117 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:25.137620 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:25.139509 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:25.247730 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bbb9d-e841-4230-952f-51b0aa90d794 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:25.268453 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:25.338498 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:25.339048 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:25.339340 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:25.339908 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:25.341445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:25.354917 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:25.356761 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:25.369238 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1438/2857] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:33.463732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be72c-79f6-458f-94c8-1598e3305b9b None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:33.463732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be72c-79f6-458f-94c8-1598e3305b9b None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:33.463732 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be72c-79f6-458f-94c8-1598e3305b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:33.512677 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-899be72c-79f6-458f-94c8-1598e3305b9b None None] MFA Rules not processed for user `364e084e459d4263815ab59720d039e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:33.513918 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-899be72c-79f6-458f-94c8-1598e3305b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:33.580008 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1420/2858] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 18:00:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:33.621322 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:33.622760 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:33.713808 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd74819-ee17-4a84-aec5-53eba830d7ce service nova] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:33.718893 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215e3c6-1148-4702-a83e-e2e145bf2916 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e698e658de56495e8d501ca6dfd75623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:33.719493 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215e3c6-1148-4702-a83e-e2e145bf2916 service swift] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:33.719774 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215e3c6-1148-4702-a83e-e2e145bf2916 service swift] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:33.720050 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215e3c6-1148-4702-a83e-e2e145bf2916 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:33.726246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=100 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:33.729996 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215e3c6-1148-4702-a83e-e2e145bf2916 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:33.731589 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215e3c6-1148-4702-a83e-e2e145bf2916 service swift] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:33.740398 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1439/2859] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 18:00:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 18:00:42.151630 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899be72c-79f6-458f-94c8-1598e3305b9b None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:42.153116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-899be72c-79f6-458f-94c8-1598e3305b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:42.222422 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899be72c-79f6-458f-94c8-1598e3305b9b None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:42.227206 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:42.227764 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:42.228040 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:42.228311 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] PATH_INFO: `/v3/users/196ffd853b234fce845f359484d4b5e2` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:42.229439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=196ffd853b234fce845f359484d4b5e2)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:42.231375 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:42.417488 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196ffd853b234fce845f359484d4b5e2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:42.511479 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196ffd853b234fce845f359484d4b5e2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:42.512020 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196ffd853b234fce845f359484d4b5e2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:42.519661 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196ffd853b234fce845f359484d4b5e2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:42.520354 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196ffd853b234fce845f359484d4b5e2'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:42.528376 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:42.537938 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1421/2860] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:42 2022] DELETE /identity/v3/users/196ffd853b234fce845f359484d4b5e2 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:42.857672 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215e3c6-1148-4702-a83e-e2e145bf2916 service swift] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:42.869698 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215e3c6-1148-4702-a83e-e2e145bf2916 service swift] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:42.930288 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:42.930924 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:42.931209 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:42.931485 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] PATH_INFO: `/v3/projects/193c5ec786e74f138b2fc58ab35367ff` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:42.932654 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=193c5ec786e74f138b2fc58ab35367ff)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:42.934619 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:43.073141 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193c5ec786e74f138b2fc58ab35367ff', 'request_id': 'req-465f6a96-dbef-4544-adb1-f8de729cc6f8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.146169 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193c5ec786e74f138b2fc58ab35367ff', 'request_id': 'req-465f6a96-dbef-4544-adb1-f8de729cc6f8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.153026 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193c5ec786e74f138b2fc58ab35367ff', 'request_id': 'req-465f6a96-dbef-4544-adb1-f8de729cc6f8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.163915 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] The token cache is being invalidate because project 193c5ec786e74f138b2fc58ab35367ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:43.164314 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.165019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.166560 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1440/2861] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:42 2022] DELETE /identity/v3/projects/193c5ec786e74f138b2fc58ab35367ff => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:43.170747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:43.172182 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:43.185699 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d88a46-12ad-4a72-92d7-235ef3f798cc admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:43.246526 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:43.247220 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:43.247501 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:43.247777 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] PATH_INFO: `/v3/users/813f01157f08489e8f602ba953cabd88` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:43.248921 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=813f01157f08489e8f602ba953cabd88)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:43.250907 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:43.438879 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813f01157f08489e8f602ba953cabd88'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.549876 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813f01157f08489e8f602ba953cabd88'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.550533 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813f01157f08489e8f602ba953cabd88'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.558749 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813f01157f08489e8f602ba953cabd88'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.559393 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df8c3869-7c10-4404-a706-3b1115f9b1ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813f01157f08489e8f602ba953cabd88'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:43.578244 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1422/2862] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:43 2022] DELETE /identity/v3/users/813f01157f08489e8f602ba953cabd88 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:43.846465 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:43.848613 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:43.862448 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f6a96-dbef-4544-adb1-f8de729cc6f8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:43.922688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:43.923324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:43.923604 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:43.923877 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] PATH_INFO: `/v3/projects/97dcf1650ae04d40aa09583b93e4a8e9` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:43.925001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97dcf1650ae04d40aa09583b93e4a8e9)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:43.927721 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:44.027446 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:44.044205 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97dcf1650ae04d40aa09583b93e4a8e9', 'request_id': 'req-b25dba4d-3600-4399-8266-f8465b8b5979'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:44.111102 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97dcf1650ae04d40aa09583b93e4a8e9', 'request_id': 'req-b25dba4d-3600-4399-8266-f8465b8b5979'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:44.118089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97dcf1650ae04d40aa09583b93e4a8e9', 'request_id': 'req-b25dba4d-3600-4399-8266-f8465b8b5979'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:44.128744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] The token cache is being invalidate because project 97dcf1650ae04d40aa09583b93e4a8e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:44.129031 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:44.129822 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-b25dba4d-3600-4399-8266-f8465b8b5979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:44.131329 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1441/2863] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:43 2022] DELETE /identity/v3/projects/97dcf1650ae04d40aa09583b93e4a8e9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:44.136740 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221431fd-2118-4c04-b26e-a9f1bf847dfe None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.137029 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221431fd-2118-4c04-b26e-a9f1bf847dfe None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.137310 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221431fd-2118-4c04-b26e-a9f1bf847dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.157580 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-221431fd-2118-4c04-b26e-a9f1bf847dfe None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:44.158843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-221431fd-2118-4c04-b26e-a9f1bf847dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:44.218386 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1423/2864] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 18:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:44.226217 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8571c-5ed3-4339-959f-acf889af3b3e None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.226502 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8571c-5ed3-4339-959f-acf889af3b3e None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.226782 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8571c-5ed3-4339-959f-acf889af3b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.246625 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-e9c8571c-5ed3-4339-959f-acf889af3b3e None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:44.247825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e9c8571c-5ed3-4339-959f-acf889af3b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:44.256339 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1442/2865] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 18:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:44.260781 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221431fd-2118-4c04-b26e-a9f1bf847dfe None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:44.270638 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221431fd-2118-4c04-b26e-a9f1bf847dfe None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:44.275478 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:44.276068 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.276347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.276619 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.277260 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:00:44.277616 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:44.279189 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:44.285919 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:44.290933 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1424/2866] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 18:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:00:44.295294 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8571c-5ed3-4339-959f-acf889af3b3e None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:44.305353 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8571c-5ed3-4339-959f-acf889af3b3e None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:44.309949 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738d74b-c315-418a-a2a2-3b33e19e20f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:44.310540 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738d74b-c315-418a-a2a2-3b33e19e20f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.310816 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738d74b-c315-418a-a2a2-3b33e19e20f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.311089 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738d74b-c315-418a-a2a2-3b33e19e20f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.311710 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738d74b-c315-418a-a2a2-3b33e19e20f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:44.313586 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738d74b-c315-418a-a2a2-3b33e19e20f7 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:44.416825 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1443/2867] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:44 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:44.421544 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:44.432611 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c934-9822-4837-885d-e9febabced4e admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:44.490788 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e067480-623b-4b45-9135-2183571fc61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:44.491394 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067480-623b-4b45-9135-2183571fc61f admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.491676 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067480-623b-4b45-9135-2183571fc61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.491952 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067480-623b-4b45-9135-2183571fc61f admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.492584 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067480-623b-4b45-9135-2183571fc61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:44.494566 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067480-623b-4b45-9135-2183571fc61f admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:44.609335 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1425/2868] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:44 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:44.613576 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738d74b-c315-418a-a2a2-3b33e19e20f7 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:44.624772 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738d74b-c315-418a-a2a2-3b33e19e20f7 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:44.629694 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251c76-f76a-4af0-afde-a327ebd73356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:44.630331 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251c76-f76a-4af0-afde-a327ebd73356 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.630615 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251c76-f76a-4af0-afde-a327ebd73356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.630892 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251c76-f76a-4af0-afde-a327ebd73356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.631538 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03251c76-f76a-4af0-afde-a327ebd73356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:44.633396 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03251c76-f76a-4af0-afde-a327ebd73356 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:44.644148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1444/2869] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:44.651980 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e067480-623b-4b45-9135-2183571fc61f admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:44.663012 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e067480-623b-4b45-9135-2183571fc61f admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:44.667697 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1690f-18cc-4487-889d-4a31b56e9027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:44.668286 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1690f-18cc-4487-889d-4a31b56e9027 admin admin] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.668568 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1690f-18cc-4487-889d-4a31b56e9027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.668843 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1690f-18cc-4487-889d-4a31b56e9027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.669483 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1690f-18cc-4487-889d-4a31b56e9027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:44.671158 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1690f-18cc-4487-889d-4a31b56e9027 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:44.682662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1426/2870] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:44.686811 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251c76-f76a-4af0-afde-a327ebd73356 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:44.697246 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251c76-f76a-4af0-afde-a327ebd73356 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:44.701797 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:44.702467 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.702749 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.703027 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] PATH_INFO: `/v3/projects/17a2ab43b0e64e888af68fc09725ef8d/users/f61b40f60c594f3299484e03ab34d19b/roles/fff136f214f2428895daf10483c542d1` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.717880 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a2ab43b0e64e888af68fc09725ef8d, user_id=f61b40f60c594f3299484e03ab34d19b, role_id=fff136f214f2428895daf10483c542d1)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:44.720641 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:44.796714 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1445/2871] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:44 2022] PUT /identity/v3/projects/17a2ab43b0e64e888af68fc09725ef8d/users/f61b40f60c594f3299484e03ab34d19b/roles/fff136f214f2428895daf10483c542d1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:44.801830 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89806ce-5585-4a18-a61a-d71adf2caa88 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.802146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89806ce-5585-4a18-a61a-d71adf2caa88 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.802418 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89806ce-5585-4a18-a61a-d71adf2caa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.821523 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-f89806ce-5585-4a18-a61a-d71adf2caa88 None None] MFA Rules not processed for user `9c52ecdd884449e5b0fb195998b79885`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:44.822778 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-f89806ce-5585-4a18-a61a-d71adf2caa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:44.883287 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1427/2872] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 18:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:44.892910 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:44.895412 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:44.954307 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d424f-c5e6-42cd-aa4e-2a0ee1ec9d78 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:44.958688 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e1996-33c3-464b-b8e6-980642bf09ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:44.959295 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e1996-33c3-464b-b8e6-980642bf09ab service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:44.959580 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e1996-33c3-464b-b8e6-980642bf09ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:44.959858 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e1996-33c3-464b-b8e6-980642bf09ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:44.969202 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e1996-33c3-464b-b8e6-980642bf09ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:44.970668 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e1996-33c3-464b-b8e6-980642bf09ab service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:44.981446 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1446/2873] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:50.626179 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e47a1-5af9-4061-b5f1-6cebf39ccf44 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.626635 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e47a1-5af9-4061-b5f1-6cebf39ccf44 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.626744 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e47a1-5af9-4061-b5f1-6cebf39ccf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.672326 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-b57e47a1-5af9-4061-b5f1-6cebf39ccf44 None None] MFA Rules not processed for user `f61b40f60c594f3299484e03ab34d19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:50.674251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-b57e47a1-5af9-4061-b5f1-6cebf39ccf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:50.756573 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1428/2874] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:50.761971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e1996-33c3-464b-b8e6-980642bf09ab service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:50.774147 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e1996-33c3-464b-b8e6-980642bf09ab service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:50.778791 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8499bce-9a91-4c3e-bd4c-e81247d431eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:50.779387 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499bce-9a91-4c3e-bd4c-e81247d431eb admin admin] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.779664 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499bce-9a91-4c3e-bd4c-e81247d431eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.779933 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499bce-9a91-4c3e-bd4c-e81247d431eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.780548 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8499bce-9a91-4c3e-bd4c-e81247d431eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:50.782472 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8499bce-9a91-4c3e-bd4c-e81247d431eb admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:50.878735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1447/2875] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:50 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:50.884017 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e47a1-5af9-4061-b5f1-6cebf39ccf44 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:50.897867 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e47a1-5af9-4061-b5f1-6cebf39ccf44 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:50.902626 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:50.967066 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d1cb-f001-435c-8e78-4baedac81501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:50.968477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d1cb-f001-435c-8e78-4baedac81501 admin admin] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.968780 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d1cb-f001-435c-8e78-4baedac81501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.969166 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d1cb-f001-435c-8e78-4baedac81501 admin admin] PATH_INFO: `/v3/users` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.970172 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3d1cb-f001-435c-8e78-4baedac81501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:50.972145 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3d1cb-f001-435c-8e78-4baedac81501 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.067719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1429/2876] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:50 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:51.071802 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8499bce-9a91-4c3e-bd4c-e81247d431eb admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.084526 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8499bce-9a91-4c3e-bd4c-e81247d431eb admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.089422 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e4bc8-4f63-413f-9ad6-cf9b94c0f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.090123 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e4bc8-4f63-413f-9ad6-cf9b94c0f54c admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.090554 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e4bc8-4f63-413f-9ad6-cf9b94c0f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.090673 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e4bc8-4f63-413f-9ad6-cf9b94c0f54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.091344 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e4bc8-4f63-413f-9ad6-cf9b94c0f54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.092971 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e4bc8-4f63-413f-9ad6-cf9b94c0f54c admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.106552 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1448/2877] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:51.110971 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d1cb-f001-435c-8e78-4baedac81501 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.126061 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d1cb-f001-435c-8e78-4baedac81501 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.132718 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81b19e-45a0-4b54-8add-74a25b312e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.133380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b19e-45a0-4b54-8add-74a25b312e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.133719 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b19e-45a0-4b54-8add-74a25b312e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.133929 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b19e-45a0-4b54-8add-74a25b312e4b admin admin] PATH_INFO: `/v3/projects/b6574081fc5b42de9db587cabbf07bc3/users/54cd882e755e4400aa8c6a2fb11ae997/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.152901 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81b19e-45a0-4b54-8add-74a25b312e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6574081fc5b42de9db587cabbf07bc3, user_id=54cd882e755e4400aa8c6a2fb11ae997, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.155623 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81b19e-45a0-4b54-8add-74a25b312e4b admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.236022 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1430/2878] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:51 2022] PUT /identity/v3/projects/b6574081fc5b42de9db587cabbf07bc3/users/54cd882e755e4400aa8c6a2fb11ae997/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 18:00:51.246440 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e4bc8-4f63-413f-9ad6-cf9b94c0f54c admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.260101 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e4bc8-4f63-413f-9ad6-cf9b94c0f54c admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.269725 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:51.326517 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0e490-4bce-40dd-804a-fa730e829b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.327161 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0e490-4bce-40dd-804a-fa730e829b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.327445 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0e490-4bce-40dd-804a-fa730e829b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.327722 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0e490-4bce-40dd-804a-fa730e829b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.328382 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0e490-4bce-40dd-804a-fa730e829b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.330078 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0e490-4bce-40dd-804a-fa730e829b04 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.345076 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1449/2879] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:51.346753 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81b19e-45a0-4b54-8add-74a25b312e4b admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.357679 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81b19e-45a0-4b54-8add-74a25b312e4b admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.362514 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.363865 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] REQUEST_METHOD: `PUT` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.364211 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.364436 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] PATH_INFO: `/v3/projects/b6574081fc5b42de9db587cabbf07bc3/users/54cd882e755e4400aa8c6a2fb11ae997/roles/7cea93ff9a6d44b2b8f0f6b15518fee1` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.366475 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6574081fc5b42de9db587cabbf07bc3, user_id=54cd882e755e4400aa8c6a2fb11ae997, role_id=7cea93ff9a6d44b2b8f0f6b15518fee1)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.369109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.379856 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1431/2880] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:51 2022] PUT /identity/v3/projects/b6574081fc5b42de9db587cabbf07bc3/users/54cd882e755e4400aa8c6a2fb11ae997/roles/7cea93ff9a6d44b2b8f0f6b15518fee1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 18:00:56.665806 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc403-0561-4cce-8095-8c9f6bb6d365 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.666492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc403-0561-4cce-8095-8c9f6bb6d365 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.666492 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc403-0561-4cce-8095-8c9f6bb6d365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.699952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-7a1bc403-0561-4cce-8095-8c9f6bb6d365 None None] MFA Rules not processed for user `54cd882e755e4400aa8c6a2fb11ae997`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:56.701196 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-7a1bc403-0561-4cce-8095-8c9f6bb6d365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:56.780497 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1450/2881] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:56.798307 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:56.800517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:56.889682 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26948c2f-09d2-4894-b352-6ac486b2a087 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:56.897181 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4a183-9e7d-41f1-bdcd-35954de9f8c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:56.897832 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4a183-9e7d-41f1-bdcd-35954de9f8c5 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.898153 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4a183-9e7d-41f1-bdcd-35954de9f8c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.898420 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4a183-9e7d-41f1-bdcd-35954de9f8c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.960774 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:56.963299 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4a183-9e7d-41f1-bdcd-35954de9f8c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:56.964941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4a183-9e7d-41f1-bdcd-35954de9f8c5 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:56.976733 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1432/2882] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:59.927684 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91846729-6024-42f5-a7aa-c93c459acfeb None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:59.928230 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91846729-6024-42f5-a7aa-c93c459acfeb None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:59.928606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91846729-6024-42f5-a7aa-c93c459acfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:59.950819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-91846729-6024-42f5-a7aa-c93c459acfeb None None] MFA Rules not processed for user `f61b40f60c594f3299484e03ab34d19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:59.952379 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-91846729-6024-42f5-a7aa-c93c459acfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:59.962427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1451/2883] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 18:00:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:01:07.122428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=2 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:09.944537 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=61 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:16.099809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4a183-9e7d-41f1-bdcd-35954de9f8c5 service nova] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:16.153693 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4a183-9e7d-41f1-bdcd-35954de9f8c5 service nova] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:16.157248 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:16.157839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:16.158146 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:16.158417 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:16.160129 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:16.170347 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:16.171752 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:16.180387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1433/2884] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:16.952865 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91846729-6024-42f5-a7aa-c93c459acfeb None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:16.954557 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-91846729-6024-42f5-a7aa-c93c459acfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:17.029927 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91846729-6024-42f5-a7aa-c93c459acfeb None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:17.034850 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:17.035453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:17.035735 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:17.036007 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:17.037809 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:17.050720 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:17.052337 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:17.062976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1452/2885] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:26.176251 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=26 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:27.058635 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=32 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:31.450785 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:31.467504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4addc-2d4e-4f36-8405-b7a4bf39f113 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:31.470510 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20dd03dd2fa04147bba827f983a58034', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:31.471103 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:31.471380 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:31.471652 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:31.473351 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:31.486727 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:31.488291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:31.500109 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1434/2886] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:33.926565 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:33.928001 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:33.940394 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80064db-3ec3-456e-8fe9-5e2b727d008a service placement] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:33.944074 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e1c1d-ae60-4b7a-bfba-07bb4eb1e718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:33.944693 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e1c1d-ae60-4b7a-bfba-07bb4eb1e718 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:33.944976 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e1c1d-ae60-4b7a-bfba-07bb4eb1e718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:33.945248 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e1c1d-ae60-4b7a-bfba-07bb4eb1e718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:33.955567 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e1c1d-ae60-4b7a-bfba-07bb4eb1e718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:33.956997 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e1c1d-ae60-4b7a-bfba-07bb4eb1e718 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:33.968042 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1453/2887] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:41.496268 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:43.962505 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=6 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:03:26.923147 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:03:26.935717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a6370-aff6-4aeb-b6b7-d7814a2e2529 service nova] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:03:26.941517 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627062c-1607-4333-8ea7-0db7eef7b282 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:03:26.942400 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627062c-1607-4333-8ea7-0db7eef7b282 service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:03:26.942674 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627062c-1607-4333-8ea7-0db7eef7b282 service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:03:26.942941 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627062c-1607-4333-8ea7-0db7eef7b282 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:03:26.944841 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0627062c-1607-4333-8ea7-0db7eef7b282 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:03:26.957837 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627062c-1607-4333-8ea7-0db7eef7b282 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:03:26.959439 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627062c-1607-4333-8ea7-0db7eef7b282 service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:03:26.969903 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1435/2888] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:03:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:03:36.965329 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=6 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:03:45.256652 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c89b4e-b1c7-4aee-9c86-c5eb131d3825 None None] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:03:45.258081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c89b4e-b1c7-4aee-9c86-c5eb131d3825 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:03:45.258081 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c89b4e-b1c7-4aee-9c86-c5eb131d3825 None None] PATH_INFO: `/` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:03:45.260223 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1454/2889] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 18:03:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 18:03:45.266101 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aad7ac-6212-4585-8fac-8c517b1fd149 None None] REQUEST_METHOD: `POST` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:03:45.266613 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aad7ac-6212-4585-8fac-8c517b1fd149 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:03:45.266728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aad7ac-6212-4585-8fac-8c517b1fd149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:03:45.308201 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.auth.core [None req-00aad7ac-6212-4585-8fac-8c517b1fd149 None None] MFA Rules not processed for user `efc068011c14492d9ba483650cf632e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:03:45.309793 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-00aad7ac-6212-4585-8fac-8c517b1fd149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:03:45.317334 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1436/2890] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 18:03:45.332631 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c89b4e-b1c7-4aee-9c86-c5eb131d3825 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:03:45.342229 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c89b4e-b1c7-4aee-9c86-c5eb131d3825 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:03:45.346135 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d786b4-b063-46e4-9267-cb9e17f6ba82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:03:45.346744 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d786b4-b063-46e4-9267-cb9e17f6ba82 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:03:45.347022 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d786b4-b063-46e4-9267-cb9e17f6ba82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:03:45.347292 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d786b4-b063-46e4-9267-cb9e17f6ba82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:03:45.355187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d786b4-b063-46e4-9267-cb9e17f6ba82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:03:45.356606 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d786b4-b063-46e4-9267-cb9e17f6ba82 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:03:45.364681 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1455/2891] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:03:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:03:46.460076 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34c12c-a620-4911-8b28-a9ecea20b581 None None] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:03:46.460597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34c12c-a620-4911-8b28-a9ecea20b581 None None] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:03:46.460702 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34c12c-a620-4911-8b28-a9ecea20b581 None None] PATH_INFO: `/` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:03:46.462291 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1437/2892] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 18:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 18:03:46.468588 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f38b-35e4-4911-8bd7-2d297c8b4ca6 None None] REQUEST_METHOD: `POST` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:03:46.469070 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f38b-35e4-4911-8bd7-2d297c8b4ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:03:46.469324 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f38b-35e4-4911-8bd7-2d297c8b4ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:03:46.509998 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.auth.core [None req-e524f38b-35e4-4911-8bd7-2d297c8b4ca6 None None] MFA Rules not processed for user `20dd03dd2fa04147bba827f983a58034`. Rule list: `[]` (Enabled: `True`). {{(pid=81886) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:03:46.511257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e524f38b-35e4-4911-8bd7-2d297c8b4ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:03:46.521018 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1456/2893] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:03:46.531461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34c12c-a620-4911-8b28-a9ecea20b581 None None] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:03:46.542877 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34c12c-a620-4911-8b28-a9ecea20b581 None None] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:03:46.547467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5103c-157d-473d-b2cf-fbcccac6fb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:03:46.547467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5103c-157d-473d-b2cf-fbcccac6fb12 service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:03:46.547467 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5103c-157d-473d-b2cf-fbcccac6fb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:03:46.547823 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5103c-157d-473d-b2cf-fbcccac6fb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:03:46.563258 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5103c-157d-473d-b2cf-fbcccac6fb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:03:46.564704 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5103c-157d-473d-b2cf-fbcccac6fb12 service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:03:46.576776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1438/2894] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:03:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:03:56.507459 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=15 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:03:56.571949 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=15 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:05:17.204067 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524f38b-35e4-4911-8bd7-2d297c8b4ca6 None None] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:17.214350 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524f38b-35e4-4911-8bd7-2d297c8b4ca6 None None] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:17.219079 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:17.220226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:17.220226 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:17.220471 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:17.222236 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:05:17.232428 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:17.233864 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:17.242928 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1457/2895] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:05:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:05:17.517873 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5103c-157d-473d-b2cf-fbcccac6fb12 service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:17.526640 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5103c-157d-473d-b2cf-fbcccac6fb12 service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:17.530385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:17.530975 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:17.531254 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:17.531530 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:17.533250 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:05:17.547415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:17.548839 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:17.560965 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1439/2896] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:05:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:05:18.744962 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:18.753837 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23302da-b4b7-47aa-83ad-668c09243a12 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:18.757514 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efc068011c14492d9ba483650cf632e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:18.758148 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:18.758427 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:18.758697 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:18.761114 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:05:18.775442 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:18.777912 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:18.789958 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1458/2897] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:05:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:05:24.823204 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:24.836657 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d920a12-2d48-4ab8-8017-c06bbb91156e service neutron] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:24.841799 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:24.842442 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:24.842728 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:24.843002 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:24.844809 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:05:24.856878 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:24.858497 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:24.869456 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1440/2898] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:05:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 18:05:28.103987 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:28.105508 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:05:28.124792 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1ac91-c8a3-4afc-a933-9e5f485d7b69 service neutron] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:28.130413 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:28.131297 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] REQUEST_METHOD: `GET` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:28.131819 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:28.132362 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:28.134486 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:05:28.147826 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:28.149630 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:28.161287 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1459/2899] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:05:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:05:28.238142 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:28.301233 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c5ec1-3532-494d-9237-98928b2306ed service placement] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:28.305717 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:28.306385 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:28.306662 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:28.306932 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] PATH_INFO: `/v3/users/f61b40f60c594f3299484e03ab34d19b` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:28.308047 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f61b40f60c594f3299484e03ab34d19b)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:28.309991 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:28.459175 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f61b40f60c594f3299484e03ab34d19b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:28.512883 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f61b40f60c594f3299484e03ab34d19b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:28.513504 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f61b40f60c594f3299484e03ab34d19b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:28.520801 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f61b40f60c594f3299484e03ab34d19b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:28.521428 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f61b40f60c594f3299484e03ab34d19b'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:28.539660 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1441/2900] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 18:05:28 2022] DELETE /identity/v3/users/f61b40f60c594f3299484e03ab34d19b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:05:28.794453 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:28.805269 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca9c04-1f3e-40ea-9930-544b4989cce7 service placement] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:28.871132 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:28.871759 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:28.872043 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:28.872319 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] PATH_INFO: `/v3/projects/17a2ab43b0e64e888af68fc09725ef8d` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:28.873742 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a2ab43b0e64e888af68fc09725ef8d)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:28.875700 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:28.982221 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a2ab43b0e64e888af68fc09725ef8d', 'request_id': 'req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.055780 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a2ab43b0e64e888af68fc09725ef8d', 'request_id': 'req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.062404 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a2ab43b0e64e888af68fc09725ef8d', 'request_id': 'req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.073424 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] The token cache is being invalidate because project 17a2ab43b0e64e888af68fc09725ef8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:05:29.073718 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.074384 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.075904 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1460/2901] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 18:05:28 2022] DELETE /identity/v3/projects/17a2ab43b0e64e888af68fc09725ef8d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:05:29.079944 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:29.081390 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:05:29.094807 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff134a14-6049-46fc-a43d-79f7394d49c6 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:29.142589 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:05:29.151279 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:29.151905 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:29.152188 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:29.152461 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] PATH_INFO: `/v3/users/54cd882e755e4400aa8c6a2fb11ae997` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:29.153601 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cd882e755e4400aa8c6a2fb11ae997)` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:29.155593 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:29.303290 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cd882e755e4400aa8c6a2fb11ae997'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.347197 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cd882e755e4400aa8c6a2fb11ae997'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.347755 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cd882e755e4400aa8c6a2fb11ae997'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.355116 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cd882e755e4400aa8c6a2fb11ae997'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.358043 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.notifications [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cd882e755e4400aa8c6a2fb11ae997'} {{(pid=81887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.374225 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1442/2902] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 18:05:29 2022] DELETE /identity/v3/users/54cd882e755e4400aa8c6a2fb11ae997 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:05:29.604187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] Authenticating user token {{(pid=81886) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:05:29.604187 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.fernet_utils [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81886) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:05:29.614562 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6ae21-c9a3-4e08-9ca6-80ad03408ac8 admin admin] Validating token access rules against request {{(pid=81886) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:05:29.664477 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:05:29.674332 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c52ecdd884449e5b0fb195998b79885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74aebe409e204672bb76c767e1d15f99', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81886) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:05:29.674952 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:05:29.675257 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:05:29.675549 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] PATH_INFO: `/v3/projects/b6574081fc5b42de9db587cabbf07bc3` {{(pid=81886) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:05:29.676636 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6574081fc5b42de9db587cabbf07bc3)` {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:05:29.678643 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] RBAC: Authorization granted {{(pid=81886) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:05:29.792438 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6574081fc5b42de9db587cabbf07bc3', 'request_id': 'req-4e9df3c0-945d-4ace-8851-3bccdf775195'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.861019 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6574081fc5b42de9db587cabbf07bc3', 'request_id': 'req-4e9df3c0-945d-4ace-8851-3bccdf775195'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.868177 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6574081fc5b42de9db587cabbf07bc3', 'request_id': 'req-4e9df3c0-945d-4ace-8851-3bccdf775195'} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.879094 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] The token cache is being invalidate because project b6574081fc5b42de9db587cabbf07bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81886) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:05:29.879395 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.880050 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG keystone.notifications [None req-4e9df3c0-945d-4ace-8851-3bccdf775195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81886) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:05:29.881525 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: [pid: 81886|app: 0|req: 1461/2903] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 18:05:29 2022] DELETE /identity/v3/projects/b6574081fc5b42de9db587cabbf07bc3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:05:39.370477 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:05:39.875901 dv-cinder-854428-1-iscsi devstack@keystone.service[81886]: DEBUG dbcounter [-] [81886] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81886) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:08:30.901117 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] Authenticating user token {{(pid=81887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:08:30.902597 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:08:31.042747 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4de89f-111d-4c2c-a1d2-c0dee76e8b69 admin admin] Validating token access rules against request {{(pid=81887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:08:31.100483 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a3c35-d55a-4026-9101-27579ce68414 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ea1ee7219994d38a9002b3a1a46383e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dc1eb9572154f2e8de63401b760a1c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:08:31.101071 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a3c35-d55a-4026-9101-27579ce68414 service placement] REQUEST_METHOD: `GET` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:08:31.101360 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a3c35-d55a-4026-9101-27579ce68414 service placement] SCRIPT_NAME: `/identity` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:08:31.101633 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a3c35-d55a-4026-9101-27579ce68414 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:08:31.103415 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.fernet_utils [None req-861a3c35-d55a-4026-9101-27579ce68414 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:08:31.114776 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a3c35-d55a-4026-9101-27579ce68414 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:08:31.116387 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a3c35-d55a-4026-9101-27579ce68414 service placement] RBAC: Authorization granted {{(pid=81887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:08:31.126435 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: [pid: 81887|app: 0|req: 1443/2904] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 18:08:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:08:41.122221 dv-cinder-854428-1-iscsi devstack@keystone.service[81887]: DEBUG dbcounter [-] [81887] Writing DB stats keystone:SELECT=89 {{(pid=81887) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}